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The present invention relates to computer interfacing, and, at least in the 
preferred embodiments, to controlling and/or providing input to computer software by a 
combination of control information and user information disposed on physical surfaces. 
5 The inventi on has been developed primarily as a means of implementing a paper- 

based interface to publications and applications on a computer network such as the 
Internet by means of an input device, and is, to a large extent, described in relation to 
this application. However, it will be appreciated that the invention, especially in its 
broadest form, can be applied to a large number of other applications. 
10 Various aspects and features of different embodiments of the invention are 

disclosed in the accompanying documents that are respectively identified as: 

Appendix 1, Netpage, System Overview, draft version 0.5; 

Appendix 2, Netpage, Applications, draft version 0.2; 

Appendix 3, Netpage Printer, Design Description, draft version 0.3; 
15 Appendix 4, Business Implementation Procedures and Equipment, draft 

version 0.1; 

Appendix 5, Netpage Pen, Design Description, draft version 0.3; 
Appendix 6, Netpage Extensions, draft version 0.2; and 
Appendix 7, Self-Mapping Surfaces, Design Description, draft version 0.3. 
1() Various exemplary, non-limiting aspects of the invention are also described and 

tie lined in the numbered paragraphs on pages 3 to 67. 
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NPOl 

A Fagged Surface 

1 . A tagged surface including a base surface upon which is disposed a plurality of 
readable tags, each of the tags including: 

first identity data for defining a relative position of that tag; and 
second identity data for identifying the tagged surface. 

la. A tagged surface according to paragraph 1, including further tags that do not 
include both first identity data and second identity data. 
10 

2. A tagged surface according to paragraph 1 , wherein the relative position is 
defined in relation to the base surface. 

3. A tagged surface according to paragraph 1 , wherein the relative position is 
15 defined in relation to a plurality of the other tags. 

3a. A tagged surface according to any one of the preceding paragraphs, wherein the 
first identity data identifies remotely stored information defining the relative position. 
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3b. A tagged surface according to any one of paragraphs 1 to 3, wherein the first 
identity data and the second identity data together identify remotely stored information 
defining the relative position. 

4. A tagged surface according to any one of the preceding paragraphs, the base 
surface taking the form of a substrate. 

4a. A tagged surface according to paragraph 4, wherein the substrate is laminar. 

5. A tagged surface according to any one of the preceding paragraphs, wherein the 
tags are disposed at predetermined positions on the base surface. 

5a. A tagged surface according to any one of the preceding paragraphs, wherein the 
tags are disposed on the base surface within a tessellated pattern comprising a plurality 

of regions, each of the regions containing a plurality of the tags. 

5b. A tagged surface according to paragraph 5a, wherein the regions interlock with 
each other to substantially cover the base surface. 

5c. A lagged surface according to paragraph 5b, wherein the regions are all of a 
similar shape. 



10 



- 5 - 

5d. A tagged surface according to paragraph 5c, wherein the regions are triangular, 
square, rectangular or hexagonal. 

5c. A tagged surface according to any one of paragraphs 5a to 5d, wherein the tags 
are disposed stochastically within each of the regions. 

5f. A tagged surface according to paragraph 5e, wherein the first identity data of 
each of the tags includes index data indicating the region within which the tag is 
disposed and position data indicating the tag's position within that region. 

5g. A tagged surface according to any one of paragraphs 1 to 4a, wherein at least a 
plurality of the tags are disposed stochastically upon the base surface. 



5h. A tagged surface according to paragraph 5g, wherein the first identity data of 
15 each of the tags includes position data indicating the tag's position in relation to either 
the base surface or a plurality of the other tags. 

() - A ta ^d surface according to paragraph 5, wherein the tags arc disposed in a 

regular array on the base surface. 

?.n 

A lagged surface according to paragraph o. wherein the array is rerfam-Mlnr 
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A tagged surface according to paragraph 6 or 7, wherein the tags are tiled ov 
the base surface. 



9. A tagged surface according to any one of the preceding paragraphs, wherein each 
of the tags includes a common feature in addition to the second identity data. 

9a. A tagged surface according to paragraph 9, wherein the common feature is 
configured to assist finding and/or recognition of the tags by associated tag reading 
apparatus. 

9b. A tagged surface according to paragraph 9 or 9a, wherein the common features 
are stored in a data format incorporating redundancy of information. 

1 0. A tagged surface according to any one of the preceding paragraphs wherein each 
of' the tags includes one or more orientation features for enabling a rotational orientation 
of the tag being read to be ascertained. 



1 Oa A tagged surface according to paragraph 10, wherein the orientation features are 
stored in a data format incorporating redundancy of information. 

11. A tagged surface according to any one of the preceding paragraphs, wherein each 
tag includes a plurality of tag elements, the first and second identity data each being 
defined by a plurality of the elements. 



12. A tagged surface according to paragraph 1 1 , wherein each clement takes the form 
of a dot having a plurality of possible values. 

1 3. A tagged surface according to paragraph 1 2, wherein the number of possible 
values is two. 

1 4. A tagged surface according to any one of paragraphs 1 1 to 1 3. wherein when 
representing one of the possible values, the tag elements absorb, reflect or fluoresce 
electromagnetic radiation of a predetermined wavelength or range of wavelengths to a 
predetermined greater or lesser extent than the base surface. 

14a. A tagged surface according to any one of paragraphs 1 1 to 14, wherein the 
possible values of the tag elements are defined by different relative absorption, reflection 
or fluorescence of electromagnetic radiation of a predetermined wavelength or range of 
wavelengths. 

1 x A lagged surf ace according to any one of paragraphs 1 1 to 14a. wherein the tags 
are not substantially visible to an average unaided human eye under daylight or ambient 
lighting conditions. 



1 5a A tagged surface according to any one of paragraphs 1 1 to 14a, wherein the tags 
are slightly visible to an average unaided human eye under daylight or ambient lighting 
conditions. 

1 6. A tagged surface according to any one of paragraphs 1 1 to 14a, wherein the tags 
are visible to an average unaided human eye under daylight or ambient lighting 
conditions. 

1 7. A tagged surface according to any one of the preceding paragraphs, further 
including additional non-tag information disposed on the base surface. 

1 8. A tagged surface according to any one of the preceding paragraphs, wherein the 

first identity data is stored in a data format incorporating redundancy of information. 

l c >. A tagged surface according to any one of the preceding paragraphs, wherein the 
second identity data is stored in a data format incorporating redundancy of information. 

20. A tagged surface according to any one of the preceding paragraphs, wherein the 
tags are printed onto the tagged surface by means of a printer. 

2 1 A tagged surface according to paragraph 20, wherein the printer is an ink printer. 



22. A tagged surface according to paragraph 2 1 , wherein the tags arc printed using 
ink that is absorbent or reflective in the ultraviolet spectrum or the infrared spectrum. 

23. A tagged surface according to any one of paragraphs 20 to 22, wherein the printer 
also prints additional information onto the tagged surface. 

24. A tagged surface according to paragraph 23, wherein the additional information 
is printed onto the tagged surface using colored or monochrome inks. 

25. A tagged surface according to paragraph 24, wherein the additional information 
is printed onto the tagged surface using one of the following combinations of colored 
inks: 

CM Y; 

CMYK; 

CM YRCJB; and 

spot colour. 

2(> A method of producing a tagged surface according to anv one of paragraphs 1 to 
25, including the steps of: 

(a) providing a base surface; 

(b) defining common second identity data for identifying the tagged surface: 

■i'^i i- ■ i'U'iiiii;-. <]aia m»i K' 1 1 1 1 1 1 u / a K'lahw [iomimii <>[ thai tau. 
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(d) disposing a plurality of the tags on the base surface in a readable form; 
wherein steps (a) to (d) can be performed in any suitable order. 

27. A method according to paragraph 26, wherein the respective first identity data 
associated with each tag defines a position of that tag with respect to the base surface. 

28. A method according to paragraph 26, wherein the respective first identity data 
associated with each tag defines a position of that tag with respect to one or more of the 
other tags. 

29. A method according to any one of paragraphs 26 to 28, including the step (e) of 
providing a substrate upon which the tagged surface is produced, step (e) being 

performed at any suitable time in relation to step (a) to (d). 

30. A method according to any one of paragraphs 26 to 29, wherein step (d) includes 
the sub-step (da) of disposing the tags at predetermined positions on the base surface. 

3 1 . A method according to paragraph 30, wherein the sub-step (da) includes the step 
of disposing the tags in a regular array on the base surface. 

32. A method according to paragraph 30, wherein the sub-step (da) includes the step 
of disposing the tags in a rectangular array on the base surface. 



33. A method according to paragraph 3 1 or 32, wherein the step of disposing the tags 
on the base surface includes the sub-step of tiling the tags over the base surface. 

34. A method according to any one of paragraphs 26 to 33, further including the step 
of adding a common feature to the tags in addition to the second identity data. 

34a. A method according to paragraph 34, wherein the common feature is configured 
to assist location and/or recognition of the tags by associated tag reading apparatus. 

34b. A method according to paragraph 34 or 34a, wherein the common features are 
stored in a data format incorporating redundancy of information. 

35. A method according to any one of paragraphs 26 to 34, further including the step 
of providing each of the tags with one or more orientation features for enabling an 
orientation of the tag being read to be ascertained. 

35a. A method according to paragraph 35, wherein the orientation features are stored 
in a data format incorporating redundancy of information. 

36. A method according to any one of paragraphs 26 to 35, wherein each tag includes 
a plurality of tag elements, the first and second identity data being defined bv a plurality 



37. A method according to any one of paragraphs 20 to 36, wherein step (d) includ 
the sub-step of disposing the readable tags on the base surface such that the relative 
spacing of their centres is less than about 12mm. 

38. A method according to paragraph 37, wherein the relative spacing is less than 
about 3mm. 

39. A method according to paragraph 37, wherein the relative spacing is less than 
about 1mm. 

40. A tagged surface including: 
a base surface; 

one or more regions defined in relation to the base surface; 
a plurality of readable tags disposed substantially within each of the regions, 
wherein the tags in each of the regions includes identity data for identifying the region. 

41 A tagged surface according to paragraph 40, wherein the tags associated with 
each region substantially fill all of that region. 

42. A tagged surface according to paragraph 41 , wherein the readable tags within 
each region are identical to each other, but are distinct from tags in a plurality of other 
regions on the display page or on other display pages. 



43. A tagged surface according to any one of paragraphs 40 to 42, wherein the tags 
within each region are positioned stochastically within that region. 

44. A tagged surface according to any one of paragraphs 40 to 43, wherein the tags 
within each region are positioned in a regular array. 

45. A tagged surface according to any one of paragraphs 40 to 44, wherein the tags 
associated with each region are substantially uniformly distributed within that region. 

46. A tagged surface according to any one of paragraph 40 to 45, wherein the 
readable tags are disposed on the base surface such that the relative spacing of their 
centres is less than about 12mm. 

47. A tagged surface according to paragraph 46, wherein the relative spacing is less 
than about 3mm. 

48. A tagged surface according to paragraph 47. wherein the relative spacing is less 
about I mm. 

49 A method of producing a tagged surface according to any one of paragraphs 40 to 
4S, including the steps of: 

^■ |m i";' "I'c <>r more regions in iclatmn to ilu- h;is,- surface: 
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(c) defining a plurality of readable tags, each of which includes identity data for 
ldentifying one or more of the regions; and 

(d) disposing a plurality of the readable tags on the base surface, such that each tag i 
substantially within the region identified by its identity data; 

wherein steps (a) to (d) can be performed in any suitable order. 



50. A method according to paragraph 49, including the step (e) of providing a 
substrate upon which the tagged surface is produced, step (e) being performed at any 
suitable time in relation to step (a) to (d). 

51. A method according to paragraph 49 or 50, wherein step (d) includes the sub-step 
(da) of disposing the tags in a regular array on the base surface. 



52. A method according to paragraph 5 1 , wherein the sub-step (da) includes the step 

of disposing the tags in a rectangular array on the base surface. 

53. A method according to paragraph 51 or 52, wherein the step of disposing the tags 
on the base surface includes the sub-step of tiling the tags over the base surface. 

54. A method according to any one of paragraphs 49 to 53, further including the step 
of adding a common feature to the tags in addition to the identity data. 



55. A method according to paragraph 54, wherein the common feature is configured 
to assist location and/or recognition of the tags by associated tag reading apparatus. 

56. A method according to paragraph 54 or 55, wherein the common features are 
stored in a data format incorporating redundancy of information. 

57. A method according to any one of paragraphs 49 to 56, further including the step 
of providing each of the tags with one or more orientation features for enabling an 
orientation of the tag being read to be ascertained. 

58. A method according to paragraph 57, wherein the orientation features are stored 
in a data format incorporating redundancy of information. 

59. A method according to any one of paragraphs 49 to 58, wherein each tag includes 

a plurality of tag elements, the identity data being defined by a plurality of the elements. 

60. A method according to any one of paragraphs 49 to 59, wherein step (d) includes 
the sub-step of disposing the readable tags on the base surface such that the relative 
spacing of their centres is less than about 12mm. 



(>1 



A method according to paragraph 60. w herein the relative spacing is les*; than 
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62. A method according to paragraph 60, wherein the relative spacing is less about 
1 mm. 



5 
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NP02 

Interface System for Interacting with Computer Software 
(Part 1) 

1 . An interface system for interacting with computer software via a display page, 
the display page including: a base surface; and a plurality of readable tags disposed at 
corresponding plurality of positions on the base surface, each of the tags including: 
first identity data for defining a relative position of that tag; and 
second identity data for identifying a virtual page corresponding with the displ 

page; 

the system including: 

(a) a computer running one or more procedures associated with the computer 
software; 

(b) storage means for storing the virtual page corresponding with the display page, 
the virtual page having at least one virtual object associated with it; 

(c) mapping means for mapping each virtual object of the virtual page with one or 
more locations on the base surface; 

(d) processing means; 

( e) reading means for: 

selecting a first location on the base surface; 

reading one or more of the tags adjacent to the first location: and 

providing the first and second identity data associated with the read tai*s to tin* 
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(0 utilise the first and second identity data to determine: the virtual page associated 
with the display page; and location data indicative of the first location; 
(g) interface with the mapping means to ascertain whether any of the virtual objects 
associated with the virtual page is associated with the location data; and 
5 (h) in the event that one of the virtual objects is associated with the location data, 
controlling the computer software in accordance with that virtual object. 

1 a. An interface system according to paragraph 1 , wherein the processing means is 
configured to implement step (f) by: 
10 (fl ) identifying the virtual page corresponding with the display page on the basis of 
the second identity data; and 

(f2) determining location data indicative of the first location on the basis of at least 
the first identity data. 

l> lb. An interface system according to paragraph 1 or la, further including display 
means for displaying the display page to a user. 

lc An interface system according to any one of paragraphs 1 to lb. further including 
one or more surface features on the display page, at least one of the surface features 
20 having a logical, semantic and/or locational association with one or more of the virtual 
objects of the virtual page associated with the display page. 
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2. An interface system according to any one of paragraphs 1 to lc, wherein the 
reading means includes: 

imaging means for imaging one or more of the tags during selection of the first 
location and outputting imaging data; and 

converting means for converting the imaging data into first and second identity 
data of the read tag or tags. 

3. An interface system according to paragraph 2, wherein the imaging means 
includes location imaging means for: 

generating location adjustment data based on a relative position of the tag or tags 
being read in relation to an imaging field of the location imaging means; and 

utilising the location adjustment data in conjunction with the first identity data to 
generate location data for use by the processor, the location data defining the first 
location with greater precision than the spacing of the tags on the base surface. 

4. An interface system according to any one of the preceding paragraphs, wherein 

the relative position is defined in relation to the base surface. 

5. An interface system according to any one of paragraphs 1 to 3, wherein the 
relative position is defined in relation to a plurality of the other tans. 



6. An interface system according to any one of the preceding paragraphs, wherein 
the first identity data identifies remotely stored information defining the relative 
position. 

6a. An interface system according to any one of the preceding paragraphs, wherein 
the first identity data and the second identity data together identify remotely stored 
information defining the relative position. 

6b. An interface system according to any one of the preceding paragraphs, wherein at 
least a plurality of the tags are disposed stochastically upon the base surface. 

7. An interface system according to any one of the preceding paragraphs, wherein 
the tags are disposed on the base surface within a tessellated pattern comprising a 
plurality of regions, each of the regions containing a plurality of the tags. 

8. An interface system according to paragraph 7, wherein the regions interlock with 
each other to substantially cover the base surface. 

9. An interface system according to paragraph 8, wherein the regions are all of a 

similar shape. 

1 0. An interface system according to paragraph 9, wherein the regions are triangular, 
square, rectangular or hexagonal. 
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II. An interface system according to any one of paragraphs 7 to 1 0, wherein the tags 
are disposed stochastically within each of the regions. 

5 1 2. An interface system according to paragraph 1 1 , wherein the first identity data of 
each of the tags includes index data indicating the region within which the tag is 
disposed and the position of the tag within that region. 

1 3. An interface system according to paragraph 1 2, wherein the tags are disposed in a 
10 regular array on the base surface. 

14. An interface system according to paragraph 13, wherein the array is rectangular. 

15. An interface system according to paragraph 1 3 or 1 4, wherein the tags are tiled 
1 5 over the base surface. 

1 o. An interface system according to any one of the preceding paragraphs, wherein 
each of the tags includes ;i common feature in addition to the second identitv data. 

20 1 7. An interface system according to paragraph 1 6. wherein the common feature is 
configured to assist location and-'or recognition of the t;n^ hv • ■ ■■ - ■ . <••■ 
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18. An interface system according to paragraph 17, wherein the common features are 
stored in a data format incorporating redundancy of information. 

1 9. An interface system according to any one of the preceding paragraphs wherein 
each of the tags includes one or more orientation features for enabling a rotational 
orientation of the tag being read to be ascertained. 

20. An interface system according to paragraph 19, wherein the orientation features 
are stored in a data format incorporating redundancy of information. 

21. An interface system according to any one of the preceding paragraphs, wherein 
each tag includes a plurality of tag elements, the first and second identity data each being 
defined by a plurality of the elements. 

22. An interface system according to any one of the preceding paragraphs, wherein 
the first and/or second identity data is stored in a data format incorporating redundancy 
of in formation. 

23. An interface system according to any one of the preceding paragraphs, wherein 
the tags are printed onto the tagged surface by means of a printer. 



24. An interface system according to paragraph 23, wherein the printer is an ink 
printer. 



25. An interface system according to paragraph 24, wherein the tags are printed usir 
ink that is absorbent or reflective in the ultraviolet spectrum or the infrared spectrum. 

26. An interface system according to any one of paragraphs 23 to 25, wherein the 
printer also prints additional information onto the tagged surface. 

27. An interface system according to paragraph 26, wherein the additional 
information is printed onto the tagged surface using colored or monochrome inks. 

28. An interface system according to paragraph 27, wherein the additional 
information is printed onto the tagged surface using one of the following combinations 
of colored inks: 

CMY; 
CMYK; 
CMYRGB; and 
spot colour. 

29. An interface system according to any one of the preceding paragraphs, wherein 
the reading means includes a reader for reading the taus. 



3 1 . An interface system according to paragraph 30, wherein the optical reader has an 
associated capture field for capturing a predetermined area of a base surface. 

32. An interface system according to paragraph 3 1 , wherein the predetermined area 
exceeds a size of the individual tags when in at least one relative rotational orientation 
with respect thereto. 

33. An interface system according to paragraph 32, wherein the predetermined area 
exceeds the size of the individual tags in all relative rotational orientations with respect 
thereto 

34. An interface system according to paragraph 32 or 33, wherein the predetermined 
area is of sufficient size, and the tags arranged on the base surface in such a manner, that 
selection of an arbitrary first location on an operative area of the display page with the 
reader ensures that at least one full tag is captured in the capture field. 

35. An interface system according to any one of paragraphs 29 to 34, wherein the 
reading means automatically commences capturing the capture field when the reader is 
positioned adjacent the display page within a range of predetermined reading 
orientations. 



36. An interface system according to paragraph 35, wherein the reading means 
automatically commences capturing the capture field when the reader is positioned 
adjacent to, or brought into contact with, the display page within a predetermined 
reading orientation 

37. An interface system according to paragraph 35 or 36, wherein the reading means 
automatically ceases capturing the capture field when the reader is removed from 
adjacency or contact with the display page 

38. An interface system according to any one of the preceding paragraphs, wherein 
the reading means is configured to read tags at a predetermined capture rate. 

39. An interface system according to paragraph 38, further including sample storage 
means for storing a sequential plurality of location data calculated from a corresponding 
plurality of captured first identity data. 

40. An interface system according to paragraph 3'). further including timing means 
lor tune stamping at least a proportion of the location data m the sequential plurality of 
location data. 



An interface svsteni accordinu to par.irr.ipli in >. I,. .,.,,, m, 
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42. An interface stream according to any one of paragraphs 39 to 41, further 
including identity recording means for adding the second identity data, or information 
based on the second identity data, to at least a plurality of the location data in the 
sequential plurality of location data. 

5 

43. An interface system according to any one of paragraphs 39 to 42, wherein the 
location data is sampled at a rate sufficient to enable storage of a facsimile of a locus of 
the reader over the surface of the display page over time. 

10 44. An interface system according to paragraph 43, wherein the capture rate exceeds 
about 50 locational data per second. 

45. An interface system according to paragraph 44, w herein the capture rate exceeds 
about 100 locational data per second. 

15 

46. An interface system according to any one of paragraphs 39 to 45, wherein the 
location data is calculated to sufficient locational precision to enable a user's 
handwriting to be reproduced from the locational data in a recognisable form. 

20 47. An interface system according to paragraph 46, w herein the location data is 
calculated to a precision exceeding 100 dots per inch ("dpi"). 



5 
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48. An interface system according to paragraph 47, wherein the location data is 
calculated to a precision exceeding about 200 dpi. 

49. An interface system according to any one of paragraphs 39 to 48, wherein the 
sequential plurality of location data stored in the sample storage means is stored as a 
virtual object associated with the virtual page. 



50. An interface according any one of paragraphs 29 to 49. wherei n the rp^Hpr ic 
housed in a stylus. 

]() 

51. An interface system according to paragraph 50, wherein the stylus further 
includes marking means for generating a mark on the display page. 



52. An interface system according to paragraph 51, wherein the marking means takes 

15 the form of: 

a graphite pencil; 
a hall-point pen tip; 
a felt pen tip; or 
a nib pen tip. 



52a. 



An interface svstem according to paragraph 52. wherein the stvlu^ includes 



53. An interface system according to any one of paragraphs 50 to 52, wherein the 
reader incorporates a pressure sensor for sensing a pressure applied to the display page 
by the stylus. 

54. An interface system according to paragraph 53, further including pressure 
recording means for adding pressure information, based on the pressure sensed by the 
pressure sensor, to at least a plurality of the location data in the sequential plurality of 
location data. 

55. An interface system according to any one of paragraphs 50 to 54, further 
including actuation means for placing the stylus into an active mode in which one or 
more of the tags can be read. 

55a. An interface system according to paragraph 55, wherein the actuation means 
takes the form of button or other user-actuable sensor disposed on the stvlus. 

56. An interface system according to paragraph 55 when dependent upon paragraph 
53 or 54, wherein the pressure sensor is part of the actuation means. 

57. An interface system according to paragraph 56, wherein the stylus is placed into 
the active mode when the pressure sensed by the pressure sensor exceeds a 
predetermined threshold pressure. 
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58. An interface system according to any one of the preceding paragraphs, further 
including a base station, wherein the reading means includes a transmitter for 
transmitting data to the base station. 



5 59. An interface system according to paragraph 58, wherein the reading means is 
configured to act as an identity token for interacting with the base station. 

60. An interface system according to paragraph 58 or 59, wherein the reading mes 
includes identification data storage means that store data associated with the reading 
10 means. 



61. An interface system according to paragraph 60, wherein the reading means is 
configured to transmit the identification data to the base station: 

during an initial negotiation phase with the base station; 

periodically; and/or 

interspersed with other data transmitted to the base station. 



62. An interlace system according to any one of paragraphs 58 to 61, wherein the 
transmitter transmits the data by means of one or more of the following transmission 
mechanisms: 

radio frequency transmission; 



i ' i - 1 » I 1 1 v t I < i I i :M ] ] I s M o [ ] , 
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optic fiber; and 
hard-wiring. 

63. An interface system according to any one of paragraphs 58 to 62, wherein the 
5 data transmitted includes any one or more of the following: 

first identity data; 
second identity data; 
location data; 
index data; 
10 location adjustment data; and 

raw image data from the reader. 

63a. An interface system according to any one of paragraphs 58 to 63, wherein the 
data transmitted includes initial first identity data or initial location data, and subsequent 
15 delta data defining a subsequent relative displacement from the location associated with 
the initial first identity data or the initial location data. 

64. [deleted] 

^o 65. An interface system according to paragraph 63, wherein each stroke is bounded 
by commencement and cessation information. 
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66. An interlace system according to any one of paragraphs 62 to 65, wherein the 
base station includes structure storage means for storing page structure data and 
transmitted data processing means for converting the transmitted data into software 
instructions on the basis of the page structure data, wherein each of the virtual pages has 
5 associated page structure data associated with it. 



67. An interface system according to paragraph 66, wherein the page structure 
defines graphical elements and/or input elements. 

68. An interface system according to paragraph 67, wherein the graphical elements 
include any one or more of: 

text; 

graphics; and 
images. 

69. An interface system according to any one of paragraphs 66 to 68, w herein the 
input elements include one or more of: 

hyperlinks; 
form fields; 
buttons; 
checkboxes; 



signature 11 elds. 



70. An interface system according to paragraph 69, wherein the transmitted data 
processing means is configured to interpret each stroke on the basis of the stroke's 
location on the display page with respect to one or more of the input elements. 

71 . An interface system according to paragraph 70, further including text conversion 
means for converting the strokes defined by a plurality of location data into text 
characters. 

72. An interface system according to paragraph 71, wherein the text conversion 
means operates automatically to convert the strokes made by a user in a text field on the 

display page. 

73. An interface system according to paragraph 71 or paragraph 72, wherein the text 
conversion means uses any one or more of the following to assist conversion of strokes 
into text: 

the sequential plurality of location data; 
the timestamps; and/or 
the pressure information. 

74. An interface system according to any one of paragraphs 71 to 73, the system 
being configured such that, in the event that text recognised from one or more strokes 



entered into a text field is below a predetermined threshold of recognition confidence, 
some or all of the strokes from that text field, or the entire display page upon which the 
text field is defined, are forwarded to a human operator, the human operator interpreting 
the strokes and inputting text of best fit. 

74a. An interface system according to paragraph 74, wherein the human operator is 
disposed remotely from the user of the display page, and the strokes are forwarded via 
the network. 

75. An interface according to paragraph 70, further including signature recognition 
means for attempting to recognise the strokes defined by a plurality of location data as a 
signature. 

75a. An interface according to paragraph 75, wherein the signature recognition means 
operates automatically to recognise strokes as they are entered by a user in a signature 
field on the display page. 

7(>. An interlace according to paragraph 75 or 75a. wherein the signature recognition 
means is also configured to verify an identity of the reader by interfacing with the reader 
identity storage means for storing data associated with the reader. 



76a. An interface system according to paragraph 76, wherein the signature recognition 
means is configured to recognise only one or more signatures associated with a given 
reader. 

77. An interface system according to paragraph 75 or 76, wherein the signature 
recognition means utilises one or more of the following to assist in recognition of a 
signature: 

the sequential plurality of location data; 
the timestamps; and/or 
the pressure information. 

78. An interface system according to any one of paragraphs 58 to 77, wherein the 

base station includes a printer. 

79. An interface system according to paragraph 78, wherein the printer is configured 
to perform in accordance with any one of paragraphs 23 to 28. 

SO. An interface system according to paragraph 78 or 79, wherein the printer is 
configured to print one or more display pages in response to user selection of an 
associated hyperlink or button on a display page. 
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81. An interface system according to any one paragraphs 78 to 80, wherein the 
printer is configured to print one or more display pages, to which a user is subscribed, 
upon receipt of a command from a remote source. 

82. An interface system according to any one of the preceding paragraphs, wherein a 
plurality of virtual pages are stored on the storage means. 

83. An interface system according to paragraph 82. wherein the «tnr„» means ic 
located remotely from the display means. 

84. An interface system according to paragraph 83, wherein the storage means forms 
part of a network, the virtual pages stored thereon being accessible via display pages. 

85. An interface system according to paragraph 84, wherein each virtual page is 
uniquely identified within the network. 

Sf>. An interface system according to paragraph 84 or 85, wherein hyperlmked 
objects are stored on, and available to the base station via. the network. 

87. An interface system according to any one of paragraphs 75 to 77. bemu 
configured to participate in verifying a user transaction rouuirmv :1 «j.„v,t M r.. 
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(Part 2) 

1 An interface system for interacting with computer software via a display page, 

the display page including: 
a base surface; 

one or more regions defined in relation to the base surface; and 

a plurality of readable tags disposed substantially within each of the regions, the 

tags in each respective region including identity data for identifying a virtual object 

associated with that region; 

the system including: 

(a) a computer running one or more procedures associated with the computer 
so ft ware; 

(b) storage means for stonng a virtual object corresponding to one or more of the 
regions defined in relation to the display page: 

(c) processing means; and 

(d) reading means for: 

selecting a first location on the base surface; 

reading one or more of the tags adjacent the first location; and 

providing the identity data associated with the read tags to the processing means; 

wherein the processing means is configured to: 

(e) ascertain from the identity data whether any of the virtual objects is associated 
with the region from which the tags containing the identity data were read; and 

(0 in the event that one of the virtual objects is so associated with the region, 
controlling the computer software in accordance with the associated virtual object. 



1 a. An interface system according to paragraph 1 , further including display means 
for displaying the display page to a user. 

2. An interface system according to paragraph 1 or 1 a, wherein the tags associated 
with each region substantially fill all of that region. 

3. An interface system according to paraeraoh 1 or 1 a. wherein the irWit,, „ f 

— i , " * * - '«^nm.ji vjtlld \J 1 

the readable tags within each region are the same as each other, but are distinct from 
identity data of the tags in at least a plurality of other regions on the display page or on 
other display pages. 

4. An interface system according to any one of the preceding paragraphs, wherein 
the tags within each region are positioned stochastically within that region. 

5. An interface system according to any one of paragraphs 1 to 3, wherein the tags 
withm each region are positioned in a regular array. 

6. An interface system according to any one of the preceding paragraphs, wherein 
the tags associated with each region are substantially uniformly distributed within that 
renion. 
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7. An interface system according to any one of the preceding paragraphs, wherein 
each of the tags includes a common feature in addition to the second identity data. 

8. An interface system according to paragraph 7, wherein the common feature is 
5 configured to assist location and/or recognition of the tags by associated tag reading 

apparatus. 

9. An interface system according to paragraph 8, wherein the common features are 
stored in a data format incorporating redundancy of information. 

;() 

10. An interface system according to any one of the preceding paragraphs wherein 
each of the tags includes one or more orientation features for enabling a rotational 

orientation of the tag being read to be ascertained. 

5 1 1 An interface system according to paragraph 10, wherein the orientation features 
are stored in a data format incorporating redundancy of information. 

12. An interface system according to any one of the preceding paragraphs, wherein 
each tag includes a plurality of tag elements, the identity data being defined by a 

o plurality of the elements. 

13. An interface system according to any one of the preceding paragraphs, wherein 
the identity data are stored in a data format incorporating redundancy of information. 
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14. An interface system according to any one of the preceding paragraphs, wherein 
the tags are printed onto the tagged surface by means of a printer. 

15. An interface system according to paragraph 14, wherein the printer is an ink 
printer. 

16. An interface system according to paragraph 15, wherein the tags are printed using 
ink that is absorbent or reflective in the ultraviolet spectrum or the infrared spectrum. 

1 7. An interface system according to any one of paragraphs 14 to 1 6, wherein the 
printer also prints additional information onto the tagged surface. 

18. An interface system according to paragraph 1 7, wherein the additional 
information is printed onto the tagged surface using colored or monochrome inks. 

19. An interface system according to paragraph IS, wherein the additional 
information is printed onto the tagged surface using one of the following combinations 
of colored inks: 

(MY; 
CMYK; 
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20. An interface system according to any one of the preceding paragraphs, wherein 
the reading means includes a reader for reading the tags. 

5 21. An interface system according to paragraph 20, wherein the reader is an optical 
reader. 

22. An interface system according to paragraph 21, wherein the optical reader has an 
associated capture field for capturing a predetermined area of a base surface. 

10 

23. An interface system according to paragraph 22, wherein the predetermined area 
exceeds a size of the individual tags when in at least one relative rotational orientation 
with respect thereto. 

l- s 24. An interface system according to paragraph 23, wherein the predetermined area 
exceeds the size of the individual tags in all relative rotational orientations with respect 
thereto. 

25. An interface system according to paragraph 23 or 24, wherein the predetermined 
20 area is of sufficient size, and the tags arranged on the base surface in such a manner, that 
selection of an arbitrary first location on an operative area of the display page with the 
reader ensures that at least one full tag is captured in the capture field. 
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26. An interface system according to any one of paragraphs 20 to 25, wherein the 
reading means automatically commences capturing the capture field when the reader is 
positioned adjacent the display page within a range of predetermined reading 
orientations. 

5 

27. An interface system according to paragraph 26, wherein the reading means 
automatically commences capturing the capture field when the reader is positioned 
adjacent to, or brought into contact with, the display page within a predetermined 
reading orientations. 

10 

28. An interface system according to paragraph 26 or 27, wherein the reading means 
automatically ceases capturing the capture field when the reader is removed from 
adjacency or contact with the display page. 

15 29. An interface system according to any one of the preceding paragraphs, wherein 
the reading means is configured to read tags at a predetermined capture rate. 

30. An interface system according to paragraph 29, further including sample storage 
means for storing a sequential plurality of identity data. 

20 

31. An interface svstem according to paragraph 30. further includim- limim- m.-.mv 



ulent i 



data 
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32. An interface system according to paragraph 3 1 , wherein the timing means time 
stamps at least the first and last identity data in the sequential plurality of the identity 
data. 

33. An interface system according to any one of paragraphs 30 to 32, wherein the 
identity data is sampled at a predetermined rate. 

34. An interface system according to paragraph 33, wherein the capture rate exceeds 
K) about 50 identity data per second. 

35. An interface system according to paragraph 34, wherein the capture rate exceeds 
about 100 identity data per second. 

15 30. [deleted] 

37. An interface according any one of paragraphs 20 to 35, wherein the reader is 
housed in a stylus. 

-° 3S. An interface system according to paragraph 37, wherein the stylus further 
includes marking means for generating a mark on the display page. 



- 43 - 

39. An interface system according to paragraph 38, wherein the marking means takes 
the form of: 

a graphite pencil; 

a ball-point pen tip; 

a felt pen tip; or 

a nib pen tip. 



40. An interface system according to any one of f 



ncira aran 
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reader incorporates a pressure sensor for sensing a pressure applied to the display page 
10 by the stylus. 



41 . An interface system according to paragraph 40, further including pressure 
recording means for adding pressure information, based on the pressure sensed by the 
pressure sensor, to at least a plurality of the identity data in the sequential plurality of 

15 identity data. 

42. An interface system according to any one of paragraphs 37 to 41, further 
including actuation means for placing the stylus into an active mode in which one or 
more of the tags can be read. 



20 



43. An interface svstem according to paragraph 4^ wlvrrin M^* .,,-t,, 
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44. An interface system according to paragraph 42 when dependent upon paragraph 
40 or 41, wherein the pressure sensor is part of the actuation means. 

45. An interface system according to paragraph 44, wherein the stylus is placed into 
5 the active mode when the pressure sensed by the pressure sensor exceeds a 

predetermined threshold pressure. 

46. An interface system according to any one of the preceding paragraphs, further 
including a base station, wherein the reading means includes a transmitter for 

10 transmitting data to the base station. 

47. An interface system according to paragraph 46, wherein the reading means is 
configured to act as an identity token for interacting with the base station. 

15 48. An interface system according to paragraph 46 or 47, wherein the reading means 
includes identification data storage means that store data associate with the reading 
means. 

49. An interface system according to paragraph 48, wherein the reading means is 
20 configured to transmit the identification data to the base station: 
during an initial negotiation phase with the base station; 
periodically; and/or 

interspersed with other data transmitted to the base station 
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50. An interface system according to any one of paragraphs 46 to 49, wherein the 
transmitter transmits the data by means of one or more of the following transmission 
mechanisms: 

radio frequency transmission; 

infrared transmission; 

ultrasonic transmission; 

optic fiber; and 

hard-wiring. 

5 1 . An interface system according to any one of paragraphs 46 to 50, wherein the 
data transmitted includes the identity data. 



52. An interface system according to paragraph 5 1 , wherein the data transmitted to 
the base station is transmitted in the form of a stroke, the stroke including: 

timestamps indicating relative or absolute timing of one or more of the identity 
data in the sequential plurality thereof: and. or 

pressure information indicating pressure sensed by the pressure sensor. 

53. An interface system according to paragraph 52, wherein each stroke is bounded 

bv commencement and cessation mfnrm:in<Mi 
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54. An interface system according to any one of paragraphs 50 to 53, wherein the 
base station includes storage means for storing virtual objects and processing means for 
converting the transmitted data into software instructions on the basis of the virtual 
objects. 

5 

54a. An interface system according to paragraph 54, further including a plurality of 
virtual pages and/or page structures stored on the storage means. 

55. An interface system according to paragraph 54a, wherein the page structure 
() defines graphical elements and/or input elements. 

56. An interface system according to paragraph 55, wherein the graphical elements 

include any one or more of: 
text; 

5 graphics; and 

images. 

5 7. An interface system according to any one of paragraphs 54 to 56, wherein the 
input elements include one or more of the following: 
o hyperlinks; and 

buttons. 



[deleted] 
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59. An interface system according to paragraphs 46 to 58, wherein the base station 
includes a printer. 

60. An interface system according to paragraph 59, wherein the printer is configured 
to perform in accordance with any one of paragraphs 14 to 19. 

61 . An interface system according to oaraeranh 59 or 60 

wherein the "rin^r is 
configured to print one or more display pages in response to user selection of an 
associated hyperlink or button on a display page. 

62. An interface system according to any one paragraphs 59 to 61, wherein the 
printer is configured to print one or more display pages, to which a user is subscribed, 
upon receipt of a command from a remote source. 

63. An interface system according to any one of the preceding paragraphs, wherein a 
plurality of virtual objects are stored on the storage means. 

63a. An interface system according to paragraph 63, wherein the storage means is 
located remotely from the display means. 



associated u ith one ot the page structures 
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6i>. An interface system according to paragraph 63b or 64, wherein the storage means 
forms part of a network, the virtual objects stored thereon being accessible via associated 
display pages. 

66. An interface system according to paragraph 65, wherein each virtual object is 
uniquely identified within the network. 



10 



67. An interface system according to paragraph 65 or 66, wherein hyperlinked 
objects are stored on, and available to the base station via, the network. 
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NP02 
(Part 3) 

Control-information Based Interface System. 

1 . An interface system for interacting with computer software running on a computer, 
the system including: 

a plurality of physical surfaces; 

control information disposed on each of the physical surfaces, the control 
information being machine readable and substantially not visible to a human user; and 

human readable user information disposed on one or more of the surfaces; 

wherein the control information is configured such that designation thereof by a 
user causes instructions to be applied in relation to the software running on the 
computer. 

2. An interface system according to paragraph 1, w herein at least some of the user 
miormation is disposed coincident with or adjacent to some or all of the control 
information. 

v An interface system ncconlim* u^ n.'irvHM-ripi" 1 ^ \^>r t >\^ ii^.. t,., ■ 
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information result in operation of the computer in a manner that corresponds with the 
meaning of that user information as would be understood by a human user. 

3a. An interface system according to paragraph 3, wherein the user information takes 
the form of any combination of: 
text; 

graphics; 
images; 
icons; and 
hypertext links. 

4. An interface system according to any one of the preceding paragraphs, wherein the 
control information takes the form of one or more control tags. 

4a. An interface system according to paragraph 4, wherein each of the control tags 
defines a discrete bundle of the control information. 

4b. An interface system according to paragraph 4 or 4a, wherein each of the control 
tags has a predetermined spatial extent on the surface. 

4c. An interface system according to any one of paragraphs 4 to 4b, wherein a user 
designates control information by designating one or more of the control tags. 
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4d. An interface system according to any one of paragraphs 4 to 4c, wherein the tags 
take the form of bar codes. 

4e. An interface system according to paragraph 4d, wherein the bar codes are two- 
dimensional bar codes. 

5. An interface system according to any one of paragraphs 4 to 4c, wherein each 
control tag defines control informati on that is unique compared to the control 
information defined by other control tags on the same surface. 

6. An interface system according to paragraph 5, wherein the unique control 
information associated with each control tag identifies a position of that control tag on 
the surface. 

7. An interface system according to paragraph 6, wherein the position is identified 
relative to the surface itself 

S. An interface system according to paragraph 0, w herein the position is identified 
relative to other control tags on the same surface. 

Ha An interface svstcm according to any nnr nf tv.r-i. »* ->nii. 1 ^ i - . ■ ■ i 
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8b. An interface system according to paragraph 8a, wherein the common feature is 
configured to assist finding and/or recognition of the control tags by associated control 
tag reading apparatus. 

8c. An interface system according to paragraph 8a or 8b, wherein the common 
features are stored in a data format incorporating redundancy of information. 

9. An interface system according to any one of paragraph 4 to 8c, wherein each of 
the control tags includes one or more orientation features for enabling a rotational 
orientation of the control tag being read to be ascertained. 

10. An interface system according to paragraph 9, wherein the orientation features 
are stored in a data format incorporating redundancy of information. 

11. An interface system according to any one of paragraphs 4 to 10, wherein each 
control tag is defined by a plurality of control tag elements, the control data being 
encoded in and defined by the control tag elements. 

12. An interface system according to paragraph 1 1, wherein each element takes the 
form of a dot having a plurality of possible values. 

13. An interface system according to paragraph 12, wherein the number of possible 
values is two. 
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14. An interface system according to any one of paragraphs 1 1 to 13, wherein when 
representing one of the possible values, the control tag elements absorb, reflect or 
fluoresce electromagnetic radiation of a predetermined wavelength or range of 
5 wavelengths to a predetermined greater or lesser extent than the base surface. 

14a. An interface system according to any one of paragraphs 1 1 to 14, wherein the 
possible values of the control tag elements are defined by different relative absorption, 
reflection or fluorescence of electromagnetic radiation of a predetermined wavelength oi 
10 range of wavelengths. 



15. An interface system according to paragraph 6, wherein the control data identifies 
remotely stored information defining the relative position. 

16. An interface system according to any one of the preceding paragraphs, wherein the 
control information includes surface identification information that identifies the surface 
upon which the respective control information is disposed. 

1 7. An interlace system according to any one of paragraphs 4 to 16, wherein the 
control information includes surface identification information that identifies the surface 
upon which the respective control information ^ r1t\- T v^-J m.j >t i : * ... i - 
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1 8. An interface system according to any one of paragraphs 4 to 4c, wherein the 
control tags within each region include common control information that is distinct from 
the control information of control tags in at least some other regions. 

5 1 9. An interface system according to any one of the preceding paragraphs, wherein at 
least some of the surfaces are defined by respective pages. 

20. An interface system according to paragraph 19, wherein the pages are paper or any 
other planar and/or laminar substrate. 

10 

21. An interface system according to any one of the preceding paragraphs, wherein 
the control information is printed onto the physical surfaces by means of a printer. 

22. An interface system according to paragraph 21 , wherein the printer is an ink 
15 printer. 

23. An interface system according to paragraph 22, wherein the control tags are 
printed using ink that is absorbent or reflective in the ultraviolet spectrum or the infrared 
spectrum. 

20 

24. An interface system according to any one of paragraphs 21 to 23 when dependent 
upon paragraph 2, wherein the printer also prints the user information onto the physical 
surfaces. 



25. An interface system according to paragraph 24, wherein the user information is 
printed onto the physical surfaces using colored or monochrome inks. 

26. An interface system according to paragraph 25, wherein the user information is 
printed onto the physical surfaces using one of the following combinations of colored 
inks: 

CMY; 
CMYK; 
CMYRGB; and 
spot colour. 

27. An interface system according to any one of the preceding paragraphs, further 
including a reader, the reader being operable by a user to designate the control 

in formation. 

2cS. An interlace system according to paragraph 27, wherein the reader is configured to 
not read any user information on the surface. 



29. [deleted] 
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30. An interface system according to any one of paragraphs 27 to 28, wherein the 
reader includes a scanner for reading the control information when the reader is placed 
into a reading position with respect to the surface. 

31. An interface system according to any one of paragraphs 27 to 30, wherein the 
reader is an optical reader. 

32. An interface system according to paragraph 3 1 , wherein the optical reader has an 
associated capture field for capturing a predetermined area of a base surface. 

33. An interface system according to paragraph 32 when dependent upon paragraph 
4. wherein the predetermined area exceeds a size of the individual control tags when in 
at least one relative rotational orientation with respect thereto. 

34. An interface system according to paragraph 33, wherein the predetermined area 
exceeds the size of the individual control tags in all relative rotational orientations with 
respect thereto . 

35. An interface system according to paragraph 33 or 34, wherein the predetermined 
area is of sufficient size, and the control tags arranged on the base surface in such a 
manner, that selection of an arbitrary first location on an operative area of the surface 
with the reader ensures that at least one full control tag is captured in the capture field. 
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36. An interface system according to any one of paragraphs 30 to 35, wherein the 
reading means automatically commences capturing the capture field when the reader is 
positioned adjacent the surface a range of predetermined reading orientations. 

37. An interface system according to paragraph 36, wherein the reading means 
automatically commences capturing the capture field when the reader is positioned 
adjacent to, or brought into contact with, the surface within a predetermined reading 
orientation 

38. An interface system according to paragraph 36 or 37, wherein the reading means 
automatically ceases capturing the capture field when the reader is removed from 
adjacency or contact with the surface. 

39. An interface system according to any one of paragraph 33 to 38, wherein the 
reading means is configured to read control tags at a predetermined capture rate. 

40. An interface system according to paragraph 39, further including sample storage 
means for storing a sequential plurality of location data calculated from a corrcspondinu 
plurality of captured control information. 

41. An interface system according to parn'raph -in inriii,., i . j;.. , 



location data. 
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42. An interface system according to paragraph 41 , wherein the timing means time 
stamps at least the first and last location data in the sequential plurality of location data. 

5 43. [deleted] 

44. An interface system according to any one of paragraphs 40 to 42, wherein the 
location data is sampled at a rate sufficient to enable storage of a facsimile of a locus of 
the reader over the surface over time. 

10 

45. An interface system according to paragraph 44, wherein the capture rate exceeds 
about 50 locational data per second. 

46. An interface system according to paragraph 45, wherein the capture rate exceeds 
15 about 100 locational data per second. 

47. An interface system according to any one of paragraphs 40 to 46, wherein the 
location data is calculated to sufficient locational precision to enable a user's 
handwriting to be reproduced from the locational data in a recognisable form. 

20 

48. An interface system according to paragraph 47, wherein the location data is 
calculated to a precision exceeding 100 dots per inch ("dpi"). 
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49. An interface system according to paragraph 48, wherein the location data is 
calculated to a precision exceeding about 200 dpi. 

50. An interface system according to any one of paragraphs 40 to 49, wherein the 
sequential plurality of location data stored in the sample storage means is stored as a 
virtual object associated with the surface upon which the reader is being used. 

5 1 . An interface according any one of paragraphs 30 to 49, wherein the reader is 
housed in a stylus. 

5 1 a. An interface system according to paragraph 50, wherein the stylus further 
includes marking means for generating a mark on the surface. 



52. An interface system according to paragraph 51a, wherein the marking means 
15 takes the form of: 

a graphite pencil; 

a ball-point pen tip; 

a felt pen tip; 

a nib pen tip; or 
20 a ink-jet printer. 



selection means lor selectively engaging the marking means. 



- 60 - 



53. An interface system according to any one of paragraphs 50 to 52, wherein the 
reader incorporates a pressure sensor for sensing a pressure applied to the surface by the 
stylus. 

5 

54. An interface system according to paragraph 53, further including pressure 
recording means for appending pressure information, based on the pressure sensed by the 
pressure sensor, to at least a plurality of the location data in the sequential plurality of 
location data. 

10 

55. An interface system according to any one of paragraphs 50 to 54, further 
including actuation means for placing the stylus into an active mode in which one or 
more of the control tags can be read. 

15 55a. An interface system according to paragraph 55, wherein the actuation means 
takes the form of button or other user-actuable sensor disposed on the stylus. 

56. An interface system according to paragraph 55 when dependent upon paragraph 
53 or 54, w herein the pressure sensor is part of the actuation means. 

20 

57. An interface system according to paragraph 56, wherein the stylus is placed into 
the active mode when the pressure sensed by the pressure sensor exceeds a 
predetermined threshold pressure. 
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58. An interface system according to any one of the preceding paragraphs, furtli 
including a base station, wherein the reading means includes a transmitter for 
transmitting data to the base station. 



59. An interface system according to paragraph 58, wherein the reading means is 
configured to act as an identity token for interacting with the base station. 

60. An interface system according to paragraph 58 or 59, wherein the reading me; 
includes identification data storage means that store data associated with the reading 
means. 



61 . An interface system according to paragraph 60, wherein the reading means is 
configured to transmit the identification information to the base station: 

during an initial negotiation phase with the base station; 
periodically; and/or 

interspersed with other data transmitted to the base station. 

62. An interface system according to any one of paragraphs 58 to 61, wherein the 
transmitter transmits the data by means of one or more of the following transmission 
mechanisms: 



ml raivd transmission. 
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ultrasonic transmission; 
optic fiber; and 
hard-wiring. 

5 63. An interface system according to any one of paragraphs 58 to 62, wherein the 
data transmitted includes any one or more of the following: 
identification data; 
control information; 
location data; 
10 index data; 

raw image data from the reader. 

63a. An interface system according to any one of paragraphs 58 to 63, wherein the 
data transmitted includes initial control information or initial location data, and 
15 subsequent delta data defining a subsequent relative displacement from the location 
associated with the initial first control information or the initial location data. 

64. [deleted] 

20 65. An interface system according to paragraph 63, wherein the transmitted data is 
transmitted as a stroke, each stroke being bounded by commencement and cessation 
information. 
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66. An interface system according to any one of paragraphs 62 to 65, wherein the 
base station includes structure storage means for storing page structure data associated 
with one or more of the surfaces, and transmitted data processing means for converting 
the transmitted data into software instructions on the basis of the page structure data. 

67. An interface system according to paragraph 66, wherein the page structure data 
defines graphical elements and/or input elements. 

68. An interface system according to paragraph 67, wherein the graphical elements 
include any one or more of: 

text; 

graphics; and 
images. 



69. An interface system according to any one of paragraphs 66 to 68, wherein the 
input elements include one or more of; 

hyperlinks; 

form fields; 

buttons; 

checkboxes; 

text fields; 



signature fields. 
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70. An interface system according to paragraph 69, wherein the transmitted data 
processing means is configured to interpret each stroke on the basis of the stroke's 
location on one of the surfaces with respect to one or more of the input elements. 

71 . An interface system according to paragraph 70, further including text conversion 
means for converting the strokes defined by a plurality of location data into text 
characters. 

72. An interface system according to paragraph 71, wherein the text conversion 
means operates automatically to convert the strokes made by a user in a text field on one 
of the surfaces. 

73. An interface system according to paragraph 71 or paragraph 72, wherein the text 
conversion means uses any one or more of the following to assist conversion of strokes 
into text: 

the sequential plurality of location data; 
the timestamps; and/or 
the pressure information. 

74. An interface system according to any one of paragraphs 71 to 73, the system 
being configured such that, in the event that text recognised from one or more strokes 
entered into a text field is below a predetermined threshold of recognition confidence, 
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some or all of the strokes from that text field, or from the entire surface upon which the 
text field is defined, are forwarded to a human operator, the human operator interpreting 
the strokes and inputting text of best fit. 

5 74a. An interface system according to paragraph 74, wherein the human operator is 
disposed remotely from the user of the surface, and the strokes are forwarded via the 

network. 



75. An interface according to paragraph 70, further including signature recognition 
10 means for attempting to recognise the strokes defined by a plurality of location data as a 

signature. 



75a. An interface according to paragraph 75, wherein the signature recognition means 
operates automatically to recogn.se strokes made by a user ,n a signature field on the 

15 surface. 

76. An interface according to paragraph 75 or 75a, wherein the signature recognition 
means is also configured to verify an identity of the reader by interfacing with the reader 
identity storage means for storing data associated with the readc 



ler. 

20 

76a. An interface system according to paragraph 76. whrrnn th.- — . 



t cad 
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77. An interface system according to paragraph 75 or 76, wherein the signature 
recognition means utilises one or more of the following to assist in recognition of a 
signature: 

5 the sequential plurality of location data; 

the timestamps; and/or 
the pressure information. 

78. An interface system according to any one of paragraphs 58 to 77, wherein the 
10 base station includes a printer. 

79. An interface system according to paragraph 78, wherein the printer is configured 
to perform in accordance with any one of paragraphs 20 to 26. 

15 80. An interface system according to paragraph 78 or 79, wherein the printer is 
configured to print control information and/or user information onto a surface in 
response to user selection of an associated hyperlink or button on another surface. 

80a. An interface system according to paragraph 78 or 79, wherein the printer is 
20 connected to a data storage means containing a plurality of virtual pages, each of the 
virtual pages being defined by control information and user information, the system 
being configured such that designation by a user of predetermined control information 
on a particular physical surface causes the control information and user information from 
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an associated one or more of the virtual pages to be printed onto respective additional 
physical surfaces. 



80b. An interface according to 80a, wherein designation of at least some of the control 
5 information on one or more of the additional physical surfaces causes the control 

information and user information from an associated one or more of the virtual pages to 
be printed onto further respective additional physical surfaces. 

81 . An interface system according to any one paragraphs 78 to 80, wherein the 
10 printer is configured to print control information and/or user information onto one or 

more surfaces upon receipt of a command from a remote source. 

82. An interface according to any one of paragraphs 80 to 81, wherein the physical 
surfaces are paper or any laminar sheet which can be printed upon. 

15 

83. An interface system according to any one of paragraphs 75 to 77, being 
configured to participate in verifying a user transaction requiring a signature. 

DATED this 1st day of December, 1999 
20 SILVERBROOK RESEARCH PTY LTD 

Attorney: JEFFREY B. SWEETMAN 
Fellow Institute of Patent and Trade Mark Attorneys of Australia 
of Baldwin Shi i s ion Waters 



Netpage 

System Overview 

draft version 0.5, 25 November 1999 




Confidential 



draft vO 5 



Document History 



Version 

0 5d 

0 4d 


Date 

25 November 1999 
20 October 1999 


Authors 

Paul Lapstun 

Paul Lapstun 
Kia Silverbrook 
Simon Walmsley 


Details 

Expanded description of television info- 
tainment handling 

Expanded signature handling protocol 
Added application detail 


0 3d 


7 September 1999 


Paul Lapstun 
Kia Silverbrook 


Split Netpage System Design Description 
into multiple documents 


0 2d [ 

1 


1 June 1999 


Paul Lapstun 
Kia Silverbrook 


Refined object models 
Added application detail 
Initial draft issue 



Silverbrook Research Netpage System Overview draft vO 5 



Contents 

Overview 1 

1 Introduction 2 

1.1 Publishing to Netpage Printers 3 

2 The Demise of Paper 5 

3 News and Advertising Trends 7 

3 1 Newspapers 7 

3 2 Online News Delivery 8 

3 3 Online Advertising g 

3.4 Online Classified Advertising 10 

4 News Personalization n 

Architecture 14 

5 Netpage System Architecture 1 5 

5 1 The Internet 1 5 

5 2 Netpages and Netpage Documents 16 

5 3 The Netpage Network 17 

5 4 The Netpage Printer 18 

5 5 The Netpage Pen 21 

5 6 Netpage Interaction pp 

5.7 Forms 23 

5.8 Standard Features of Netpages 24 

5 9 The Help System 24 

6 Personalized Publication Model 26 

6. 1 Editorial Personalization 26 

6 2 Advertising Localization and Targeting 27 

6 3 User Profiles 29 

6 4 Intelligent Page Layout 30 

6 5 Document Format 30 

6.6 Document Distribution 31 

6 7 On-Demand Documents 32 

7 Security 34 

7 1 Cryptography 34 

7 2 Netpage Printer Security v-. 

7 .3 Netpage Pen Security >c 

7.4 Secure Documents 37 

7 5 Non Repudiation 37 

8 Electronic Commerce Model 38 

8 1 Secure Electronic Transaction (SET) 38 

8 2 SET Payments 3 8 

8 3 Micro-Payments 3q 

8 4 Transactions 39 

Applications and Business Models 40 

9 Applications 41 



^5 rj-jvomtn-r 1 l V )' * 



Silverbrook Research M ., n3np c wctom ^ 

_ Netpage bystem Overview draft vO 5 



9 1 Personalized Subscriptions 41 

9 2 On-Demand Publications .... 42 

9 3 E-Commerce 

9.4 Communication 43 

9 5 Corporate and Government 

9.6 Personal. 4/ 

AO 

9.7 Consumer Electronics 

9.8 Advertising 50 

3 5Q 

9 9 Product Information and Registration ... 51 

9.10 Geographic Information 53 

10 Business Models 

• «•..........,..,.... ^ ^ o4 

10.1 System Principles 

10.2 Bootstrapping 

10.3 Maturity 55 

56 

References 5g 

11 Silverbrook References 60 

12 Other References ^ M 

61 



Silverbrook Research Netpage System Overview draftvO : 



Overview 



Silvertxook Research ^ Netpage System Overview draftv0 5 



Introduction 



Nctpages are pages of high-quality text, graphics and images printed on ordinary 
paper, but which work almost like interactive Web pages. Information encoded on' 
each page in invisible ink is picked up by an optically-imaging pen and transmitted 
to the network. Active "links" and "buttons" on each page can be "pressed" w ith 
the pen to request information from the network or signaF preferences to a server. 
Text written by hand on a Netpage is automatically recognized via the pen, 
allowing tonus to be filled in. Signatures recorded on a Netpage are automati- 
cally verified, allowing e-commerce transactions to be securelyauthorized. 

The pen, shown on the right, works in conjunction with a Netpage Printer, an 
Internet-connected printing appliance for home, office or mobile use. The 
pen is wireless and communicates with the Netpage Printer usinn an 
encrypted radio frequency signal. 

The Netpage Printer delivers, periodically or on demand, personalized newspapers, maga- 
zines, catalogs, brochuics and other publications, all printed at hiuh quality as interactive 
Netpages. Unlike a personal computer, the Netpage Printer, shown on the risht is an 
appliance typically wall-mounted in the kitchen or near the breakfast table i e ^the place 
where the morning new s is first consumed, and the household's point of departure for the 
day. It also comes in tabletop, desktop, portable and miniature versions. 

Netpages printed at their point of consumption combine the ease-of-use 
of paper with the timeliness and interactivity of an interactive medium. 

Netpages are crucially enabled by Memjet printing technology [2], 

which makes high-speed magazine-quality printing affordable to con- 
sumers. A Netpage publication has the physical characteristics of a tradi- 
tional newsmagazine, i.e. a set of letter-size glossy pages printed in full 
color on both sides, bound together for easy navigation and comfortable 
handling. The Netpage Printer prints 60 to 90 full-color Netpages per 
minute. 

I he Netpage Printer exploits the growing availability of broadband Inter- 
net access. Cable service is available to 95% of households in the United 
States [57], and cable modem service offering broadband Internet access 
is already available to 20% of these [9], The\'etpage Pr inter also oper- 
ates with slower connections, but with longer delivery times and lower 
image qual it\ . 

Netpage Publication Servers on the Internet deli\er print-qualit\ publica- 
tion*, to Netpage Printers. Periodical publications are delivered automati- 
cally to subscribing Netpage Printers via pointcasting and multicasting 
Internet protocols. Personalized publications are filtered and formatted 
according to individual user profiles. 




A Netpage Printer supports an> number of pens, and a pen can work with am number of 
Nci[\u*e Printers bach NVr;\f:- P.- f ■ ■ « . < •. : • ; 
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n/ed using the Netpage Pen. The Netpage Registration Server compares the Signature cap- 
tured by the Netpage l>en with a previously registered signature, allowinu it to authenticate 
the user s identity to the e-eom.nerce system. Other biometrics can also be used to verily 
identity. A version of the Netpage Pen includes fingerprint scanning verified in a similar 
way by the Netpage Registration Server. 

Although a Netpage Printer delivers periodicals such as the morning newspaper without 
user intervention. ,t never delivers unsolicited junk mail. It only delivers periodicals from 
subscribed or otherwise authorized sources. The Netpage Printer is unlike a fax machine 
or e-mail account which is visible to any junk mailer who knows the telephone number or 
email address. 

1.1 Publishing to Netpage Printers 

There are a number of advantages to publishing to Netpage Printers. The magazine quality 
of Netpage Printer output makes it a more attractive publishing and advertising medium 
than both traditional newsprint and computer screens. 

The cost of paper and ink consumption is transferred to the user. Subscription fees can be 
eliminated entirely in lieu of the user taking on this extra cost, and the user thus perceives 
a better-value product. The erratic price of newsprint is removed from publishers' profit 
equations, resulting in more stable margins. A new market for paper and ink consumables 
with its own margins, is created. 

The cos. of consumables can be selectively subsidized, for example when non-editorial 
publications such as product brochures and account statements are printed. 

Capital and maintenance expenditure on printing plant is effectivelv transferred to the 
user, although the perceived expense is small because Netpage Printers are sold at close to 
cost or given awav to encourage adoption, subsidized bv future advertising profits Main- 
tenance can also be subsidized or its cost included in a longer term serviceagreement. 

Costly physical distribution is replaced by electronic distribution via a preexisting and 
widelv subscribed network - the Internet 

Both the editorial and advertising content of publications delivered via the Netpage Net- 
work can be customized for each user. Editorial content can be personalized accordino to 
the user s profile. Advertising can be localized to the user's locality and optionalK "tar- 
geted to the user's demographic. 



A personalized publication can be a small fraction of the size of its traditional -del , v ered 
counterpart, vet contain the same amount of information relevant to the user, and in a more 
accessible form. I he user appreciates the more efficient and digestible publication. 

l ocalized advertising can be targeted to more specific localities and their associated 
demographics, and this allow s advertising space to be exploited more cfflcentlv. , e w ith 
less waste. Advertisers are constantly pressing traditional publishers for greater localiza- 
tion, something winch thev have great difficultv deliverinu cost-ef'tectiv , " 



velv 



By revealing personal information such as age, gender, marital status, income profession 
education, etc.. the user can allow the advertising to be more carefully tarueted In return 
thev can reccve greater subsidies and discounted p.oduct oilers As advert, sum become, 
more targeted, it becomes less of a nuisance and more of a service in itself 
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A though the publications per-page circulation figures fall drastically, ,he publication's 
aaual per -_,,„„ rcadcrship IS pa , ervcd flmJ „„ c , ndin| , |v > " ~ »« 

-ates for personalized delivery can exactly compensate for this. ^ ' advertising 

d A el^ S '?nn, Cl,VerCd V ? NC ' PagC NC,W ° rk has ,he dual benefits of print and online 
d U very. , rint sup rts (he , mpact of |argc . fonna( ads Qn|jne 

nation, linking, and insurability, and consequently online charging model! 

t C h?"S, Cr 3 fU ''- page , advcrtisell1t " t f»r a new car model in a newsmagazine delivered via 
I Netpage Network The advertising campaign can be national or even interna, on\ 
I ad only appears „ compatible with the user's demographic, either implied bvXi 
r J; '»°- explicitly by their persona, details. Anyone w ho requests a product _- 

churc v,a the on-ad button receives one immediately via their Netpaee Printer custom bTd 

br 1: rthe s 1 deaiers - ,f they press a particu,ar deaicr - s - bu t r : 

brochure, the dealer rece.ves a message via the system and contacts the user by telephone 
L h , C ; P u bh !' 1Cr Pr ° fit l in then0rmai by selling the advertising space, but can »i™ 

p';^„;;3 a cv ™ " c,,ck - ,hrou§h " to th * brochi,re - a,id a — 1 - 

The Netpage Network promises to be the most effective advertising medium ever con 

rarify fi neZ Tn ,h t ed ' t0rial Pnnt qU3lh >' ° f ' raditi0na ' P^ b1 -'- wih a C r 
anly finely targeted advertising, and provides a direct link between advertisino Drodu ' 

t formation, and purchasing. Added revenue from click-throuah fees and a^'merc 

;;:;;:z , ,;:; i> cven allow ^ ™ - p — ^ ^ — «™zz 
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2 The Demise of Paper 

Online publieation has many advantages over traditional paper-based publication. \ mm 
the consumer's point of view, information is available on demand, information can be nav- 
igated via hypertext links, information can be searched, and information can be automati- 
cally personalized. 

from the publisher's point of view, the costs of printing and physical distribution are elim- 
inated, and the publication becomes more attractive to the advertisers who pay for it 
because it can be targeted to specific demographics and linked to product sites. 

Online publication also has a few disadvantages. Computer screens are inferior to paper. 
At the same quality as a magazine page, an SVGA computer screen displays only about a 
fifth as much information 1 . Both CRTs and LCDs have brightness and contrast problems, 
particularly when ambient light is strong. Ink on paper, being reflective rather than emis- 
sive, is both bright and sharp in ambient light. 

l aced with reading more than the most trivial amounts of text on a screen, most people 
prefer to print it before reading it. Increasingly, online publishers are recounizirm this and 
providing information in formats suitable for printing. At one extreme this means provid- 
ing text-only versions of documents so they print efficiently, i.e. without imposing a 
screen format on the printed page; at the other extreme it means providing formatted ver- 
sions of documents - e.g. in Adobe's Portable Document Format (PDF) - so they print at 
high quality. 

I ■ditorial content is often compromised to fit the online medium and the habits of the peo- 
ple who frequent it, who tend to browse rather than read. Although powerful new advertis- 
ing models become possible, it becomes more difficult to deliver effective advertisements. 

Io trulv enable online publication, manv people envisage a universal information appli- 
ance - a lightweight portable "tablet" with a page-size touch-sensitive color display and a 
high-bandwidth wireless connection to the Internet. First proposed bv Xerox PARC's Alan 
Kay in the mid 1970s in the form of the "Dynabook", and partially realized in recent 
paperback-sized electronic books, even Bill Gates is now contldentlv predicting that such 
a device will soon augur the death of paper publications [25]. 

Io achieve low power consumption, low weight, and paper-like display quality, a bistable 
reflective display technology is required. Several candidates are now emerging from the 
labs, including Kent Display's cholesteric LCD technology (chLCD) [36]. Xerox' "Gvn- 
con" rotating ball technology [2SJ. and I Ink's eleetrophoret ic tcchnofogv [2 1.47]. 
ChLCD is arguablv closest to practical deployment [20]. 

Next-generation cellular phone networks promise 2Mbps packet switching [22], compara- 
ble to the broadband access people are getting used to in cable networks [44]. Satellite net- 
works, while offering or promising still higher speeds [2^.50,63]. require receivers 
difficult to deplov in mobile dev ices. 

Bevond the vision of the basic tablet, L Ink imagines its digital ink '•printed'" onto a num- 
ber of flexible pages hound into a book, preserving the phvsical nav iuabihtv of a 
paper- based publication, and approaching its low cost, but allow ing the pages to be rewrit- 



ssummg a nuga/ino page has an equi\alen( digital resolution of 200 continuous-lone pixels per inch 
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ton eleetroniealK in place. They optimistically predict newspapers delivered in this way 
within five years [16]. despite fundamental problems yet to be overcome [20]. 

I he advantages of a tablet are many. Unlike a desktop or notebook computer, a tablet may 
actually provide a pleasant reading experience. Unlike a paper publication, a tablet pro- 
vides intelligent access to an unlimited amount of information; its weight is not dictated 
by the amount of information it carries. More than just an information appliance, it can 
also act as a multi-purpose multimedia communications device and interactive entertain- 
ment device. 

A tablet has disadvantages too. It uses batteries which run down and have to be recharged. 
It may break when dropped or malfunction when exposed to hot coffee. It's not quite 
cheap enough to be disposable - so there's still a problem if it's misplaced or stolen, h has 
a "user interface" which has to be learned. The leading candidate display technology - 
chLCD - is still less than half as reflective (i.e. ^bright") as paper. 

The drawbacks of traditional paper-based publications have little to do with paper itself, 
and much to do with how the information gets onto the printed page. The economics of 
centralized printing and distribution prevent the kind of information selection, personal- 
ization and navigation people have come to expect from interactive electronic media such 
as the Internet. The inefheiency of printing and distributing a hundred-page newspaper to 
a customer who may read only a few pages is widely decried. 

Given a technology such as Memjet, it becomes economic to print high-quality publica- 
tions at their point of consumption rather than at their point of production. The Netpaae 
Printer leverages Memjet to deliver personalized publications to the home, gaining many 
of the advantages of online publication, while retaining the ease-of-use of high-quality 
printed ink on paper. 

Netpages and the Netpage Printer address the key problems of online publication, without 
reiving on the de\elopment and consumer acceptance of a new reading device. 
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3 News and Advertising Trends 

3.1 Newspapers 

People obtain news from a variety of sources - network and cable television, radio, daily 
newspapers, and weekly newsmagazines. In the United States, although the various news 
media are healthy and profitable, per capita news consumption is somewhat in decline as a 
new generation of young adults have less time to read and favor television entertainment 
over news ( I 7]. Yet six out often adult Americans read a newspaper every day [53]. 

The United States has about I 500 daily newspapers with a total circulation of 57 million. 
Just the top ten ''national" dailies (Wall Street Journal. USA Today, New York Times. Los 
Angeles Times, Washington Post, etc.) account for a circulation of 10 million. The major 
weekly newsmagazines (Time, Newsweek, U.S. News) have a similar (weekly) circula- 
tion of about 10 million. 

In l c )97. newspaper companies" revenue exceeded S24 billion, a five-year high, and mar- 
gins nudged 20% [39], due both to increased spending on advertising and to reduced 
prices of newsprint. 

lelevision and radio, by their nature, excel at delivering breaking news. Newspapers and 
newsmagazines, on the other hand, deliver the depth and analysis behind the headlines. 
Broadcast new s in isolation does a poor job of informing the public The more local the 
news is, the poorer the broadcast coverage, and the greater the public's dependence on 

newspapers. 

Newspaper content and packaging has evolved considerably since the 1970s. News is 
somewhat softer, news stories are shorter and more well-written, there are more feature 
articles, and there is more editorial and reader opinion. Newspapers are more structured 
1 dentil [able sections make them more accessible, and provide greater locus for advertis- 
ers. Much special-interest content has migrated from daily inclusion to weekly sections. 
1 hese cover topics such as lifesty le, personal finance, entertainment, technology, etc. The 
proportion of graphics and pictures is greater Color is widely used. Newspapers are easier 
to use and more entertaining than ever be tore, if at the expense of some "hard" news. 

Daily newspapers are growing increasingly dependent on the various wire services. A 
newspaper may excel at local and regional news, but rely on the major wires (Associated 
Press and United Press International) for national and international news, the so-called 
"supplemental" wives (I A limes Washington Post. NY limes. Scripps-Mow ard. 
Knight- Kidder In bun e. etc.) tor spec i fic strengths (and value- for-mone> ). and the interna- 
tional wires (Reuters etc ) for international perspective. A growing number of newspapers 
operate more as news aggregators than news gatherers. 

Advertising typically contributes more than 75% of newspaper and magazine revenue, 
while subscriptions contribute less than 2 5% [27,51.52], National advertising makes up 
roughly 14% of adv ertisum spending, retail advertising 46%, and classified advertising 
40% [S2]. 

Advertisers are pursuing increasingly spec i tic targeting, favoring quality newspaper read- 
ership over raw circulation [24], and using more targeted media where possible. Maga- 
zines, tor example, have more specific readerships than newspapers, free "shoppers" are 
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very localized bv their nature, while direct mailers can target demographics based on mdi- 
v idualK -categorized ZIP Codes, or databases of individuals. 

Newspapers have responded with geographically zoned editions to support local advertis- 
ing, and greater sectioning of their product. They have also expanded their page counts to 
provide more advertising scope, despite erratic newsprint prices in the 1990s [5]. 

Despite tfi is, there is ongoing conflict between newspapers' mass distribution model, and 
advertisers' need for micro-targeting [26]. This conflict, coupled with advertisers 1 desire 
for higher-quality printing of color images, is motivating a shift from run-of-press (ROP) 
advertising to inserts [52). The downside to inserts is that editorial context is lost. 



hearing the online migration of advertising, traditional news publishers from both broad- 
cast and print have ventured into Internet-based news delivery, wanting to establish a pres- 
ence at whatever cost before newcomers become entrenched. Most newspapers are still 



Online news delivery oflers a number of advantages. Breaking news can be delivered as 
soon as it happens. News can be customized for individual readers according to their pref- 
erences and geographic locations. Readers can explore stories to arbitrary depth, follow 
links to related resources, and search archival material. Readers can participate in discus- 
sion groups and contribute to opinion polls. The news itself can incorporate audio and 
video clips, and can include live transmissions, converging with broadcast. 

Online news delivery also has disadvantages. Computer screens are of limited size and 
quality compared with print. Few people enjoy reading a story of any length on a com- 
puter screen. Computers are not portable in the wide sense that a newspaper is. The news 
ma> be more timely, but the time and place iti which it can be consumed are more con- 
strained than with a newspaper. 

Despite the power of hypertext, many online readers express a preference for a linear pre- 
sentation, "where they [can] skim one section after another until the presentation [is] 
exhausted" [ 1 1 ]. Interesting!), a majority of traditional newspaper readers admit they scan 
every page in the main section of t lie newspaper [53], looking for items of interest w ithout 
necessarily know ing w hat they're looking for, and achieving some kind of closure at the 
end. Online hypertext, by contrast, is both a limitless resource and a bottomless pit 

Wfiile traditional news publishers such as 1 he New York Times can deplov full editorial 

k.onleni onlmc |<>4|. newcomers such as Yahoo tvpieallv onlv provide "raw"' news items 
sou reed tioin the w ire serv ices | IX | 

A recent survey indicates that 2 1% of the 74 million Internet users in the United States 
regularh read news online as an alternative to traditional print and broadcast sources, and 
16% obtain a major proportion of their news online [54]. More broadly, between 37% and 
<>4% ot the Internet population reads news on I me at least once a week. I lie fluctuations in 
the Injures are related to what may be happening in the news Major or breaking news sto- 
nes attract more user- - 4/i"., of' American- •. e. ^■•■\ on'v h^'.^A n il r<">. <.,-■■■ 
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news, entertainment news, and local news. As one observer puts it, all of this "sound[s] 
like the 6 o'clock news" [54]. As a reflection of these habits, the online audience share of 
national newspapers has diminished from 2 3% in 1 c W to 16% in l') { )8. while the online 
audience share of broadcast TV sites has grown. 



At its simplest, advertising alerts a motivated customer to the availability of a product, 
possibly at a competitive price. At a more sophisticated level, advertising seeks to influ- 
ence future purchasing decisions by creating brand awareness. Ultimately, advertising 
seeks to create desire for a product even when actual need is absent. 

Advertising prices are traditionally based on how many people see the advertisement, and 
their spending power in relation to the product. In practice, the more homogeneous the 
demographics of the audience, the easier it is to match to a product, and hence the higher 
the corresponding advertising cost per thousand (CPM). Broadcast media use ratings and 
times lot demographics to set advertising rates. Print media use audited circulation figures 
and sectional readership demographics. 

The simplest online advertising model is also based on how main people see the ad. 
Online this has the advantage of being based on solid numbers, since the number of 
"impressions" of a particular Web page can be counted exactly 

The specific advantage of an online ad, how ev er, is that the ad itself can measurablv cap- 
ture a sales lead by acting as a link to a product site. The product site may simply provide 
more product information in the form of specifications, pricing, and ordering details. It 
may also support immediate online ordering, thus completing the link from ad to sale. 
Beyond providing simple ad exposure, it is this measurable linking of advertisement to 
sales lead or sale which is the strength of online advertising [55j. Cost per click (Ci.C) 
charging is gaining acceptance but is still controversial. 

Beyond CLC. there exists the possibility of paying a commission to the ad host on any sale 
that actually eventuates [58 J. Amazon.com is probablv the best-known example of a com- 
pany paying commissions to other sites in this way 

The broader advantage of online advertising is that advertising can be localized and tar- 
geted arbitrarily finely; in conjunction with the publication of online content such as news. 
This is the strategy pursued by online advertising agencies such as Click Through [15]. 
which acts as the middle-man between advertisers and online content publishers. They 
expect online advertising to represent more than 10 percent of all advertising revenue b\ 
200 1. 

Since online ads are necessarily small-format, thev communicate best with motivated cus- 
tomers already on the look-out for a particular product or service. Online ads are less 
suited to building brand awareness or creating buying desires, since the real substance of 
the advertisement - the product Web site - is a click away from the initial small -format ad. 
A small-format online ad can't provide the single-hit emotional impact of a large-format 
print ad. and conversely, the online world can't support the large-format ads that print can 

So-called interstitial ads. which appear full-screen when traversing from one page of 
information to the next, go some way to providing a medium for larger- form at ads online 
( 581 I lscr resistance, however, seems to be preventing their w idespread use 
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3.4 Online Classified Advertising 

Classified advertising is indisputably suited to online delivery. Unlike their tnditioml 
pnnted counterparts, online classifieds can be easily searched, and are n< « S ^ 
constraints. I he online migration of classified advertising is considered a serio^ hrS 0 
newspapers classified advertising profits [60], and some newspapers are Cild l 
onhne presence for this reason alone. Some observers predtct as nnid'as 50% o a ifi 
advertising revenue moving online within the next ten years [72] <-i,iss.titd 

Another problem faced by newspapers, who rely on classifieds for up to 40% of advertis- 

av oT bu',M ; 15 neWCOmerS 0frcring ^ ° nlinC advertising 

way of building a venue lor non-classified advertising 
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News Personalization 

from the reader's point of view, a personalized news publication can provide more in for- 
mat ion in fewer pages. The actual form this personalization takes, however, is not neces- 
sariK obvious. 



The Mi l Media Lab's News in the future (NiF) project has been championing the concept 
of "The Daily Me"' for almost two decades [45]. Nicholas Negroponte, one of the project's 
founders, envisages a highly personalized news publication which is no longer driven by 
"what other people think is news" [49]. By way of examples close to his own needs, it 
includes news about people and places about to be encountered, and puts "the most impor- 
tant [news] of all" - a summary of e-mail - on the front page [7], Negroponte recognizes 
the need to vary the degree of personalization, advocating a higher "serendipity factor" on 
a lazy Sunday than on a working weekday. 

The opposing view holds that the value of a news publication lies precisely in its shared 
nature. It reflects the common concerns and values of a community of readers, and estab- 
lishes a baseline of expectations of what they are all supposed to know [67]. As a conse- 
quence, the publication also speaks with a consistent editorial voice and with consistent 
assumptions about the reader's level of background knowledge. Such a shared publication 
allows its readers to orient themselves in relation to their community. 

Nil "s Walter Bender answers the charge (in his own words) of "The Daily Me engender- 
ing a fragmented world populated by self-interested myopes", by stressing the possibilities 
of personalizing individual news items [6]. This can consist in varying the depth of an 
item, or supplementing it with background information, based on the reader's level of 
knowledge. It can involve interpreting information relative to the reader's background, 
such as (somew hat dubiously) making value judgements about the weather relative to the 
reader's normal home town weather, it can also be as simple as using metric rather than 
imperial units. 

I ishWrap [I 1,46]. MIT's personalized campus newspaper and NiFs latest offering, goes 
turther by creating a front page whose content represents an explicit communitv consen- 
sus. Fach tront page news item is prioritized according to the number of readers who put it 
forward for inclusion. The rest of the newspaper is still personalized according to each 
reader's profile, consisting of reader-defined sections containing topics of interest. 

I here are two implications of recognizing the shared nature of news. Firstly, some new s is 
news to everybody in a community, no matter how personalized they claim they would 
like their news to be. This implies that the community must make decisions about news 
item priority, either directly (as in Fish Wrap) or indirectly via a prow (i.e. an editor). 

Secondlv. a news item can only be properlv understood in the context of the communitv 
for which it is intended. This implies that a news item must be branded with its source 
(assuming that the source implies the intended target). As an example, it is significant 
whether a news item regarding the proof of Fermafs last theorem is branded w ith New 
Scientist or The New York l imes. To a professional mathematician, the latter implies, by 
its very existence, thai the proof is of significance bevond the scientific communitv. 

Of course, a news item must also be branded to allow its source to build and maintain its 
brand. The brand then allows the reader to infer the quality of the news item from the 
known qualitv of the source 
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Most personalization of news usc<, feuiure-hased filiering. This means that news item con- 
tent is matched to topics and keywords in the reader's pro tile. News sources tag the items 
they produce with various information to allow them to be effectively filtered. This tag- 
ging may be briefer extensive, and may include such things as news item urgency, byline, 
news category, subject(s). keyword(s). date and time, and location (33). The body text of a 
news item can also be scanned directly for keywords, but this may result in false matches 
if keywords are interpreted out of context. Items in the text such as personal names and 
locations can be tagged to reduce such ambiguity [33]. Similarly, dates, times, and mone- 
tary amounts can be tagged to allow localized presentation. 

f eature-based filtering suffers from a number of problems. Filtering based on ta»s is only 
as good as the original tagging. The latest tagging standards are only just beginning to be 
adopted [8). Filtering based on the text itself is constrained by the intelligence of the text 
parsing. If based simply on keyword matching, it can be both inaccurate, generating false 
matches because of word sense ambiguity, and imprecise, generating false mismatches 
because of a lack of inference. 

f eature-based filtering is incapable of discerning more abstract attributes such as quality, 
style, and point-of-view (unless they're indicated by tags). And since it only matches 
items anticipated by the user's profile, it is a poor generator of serendipitous finds. 

fish Wrap's front page comes into existence based on a crude form of collaborative filter- 
ing. In its broader form, collaborative filtering involves sharing recommendations (or rat- 
ings) among like-minded people [56]. This means that one person's ratings influence 
another person if and only if the two share similar interests, i.e. they have similar ratiim 
histories. Collaborative filtering overcomes many of the problems of feature-based filter- 
ing, since ratings originate with people who have digested the items in question, rather 
than from automated analysis of the items. Collaborative filtering sidesteps the issue of 
explaining win a person might like a particular item. 

Collaborative filtering has problems of its own. The system onlv works if people are will- 
ing to contribute ratings. In contributing ratings, of course, they are both doing the com- 
munity a service and tuning their own interest profiles. The statistical error in correlating 
people's interests decreases as the number of ratings increases. However, incentives may 
have to be offered to encourage people to contribute ratinus. 

Io bootstrap the accumulation of ratings for new items, an independent mechanism must 
exist to distribute them to a critical mass of people. Conversely, to bootstrap the accumula- 
tion ot interest profiles for new users, an independent mechanism must exist to distribute a 

crit ical mass of items to them 



1»» allow meaningful accumulation ot latnms. a sufficient period ot time must be allowed 
tl) elapse I his ma> contl k t with the timek delivers of items in question 

I he statistical correlation between different people's interests, represented bv their ratine 
histories, is most meaningful when the ratings applv to homogeneous items, for a set of 

heterogeneous items, collaborative filtering is best applied to homogeneous subsets. 

In a news setting, collaborative filterm-.' is best applied to te;ituie aroele- \ eunm- h 
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NaturalK. the larger a publication's readership, and the better its taste in relation to a 
potential reader, the stronger the publication's brand will appear to that reader. 

Although it's eas> to become preoccupied with automatic filtering, in reality there's more 
to editing the news than just filtering news feeds. An editor also solicits news, commis- 
sions analysis, and offers opinion, ideally ensuring that the publication offers a balanced 
and complete view of the world. 

Perhaps the most important personalization step a reader takes is in selecting a particular 
publication from a set of available publications, based on its perceived quality and rele- 
vance. 

Thus the publication's brand equates to the highest-level and most useful filter of all. 
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5 Netpage System Architecture 

5.1 The Internet 

The Internet is a worldwide collection of interconnected networks which communicate 
using the TCP/IP protocol suite [59]. A TCP/IP-based internetwork not connected to the 
Internet is often referred to as an internet (i.e. with a lower-case T). When an internet is 
deployed within an organization, it is often termed an intranet. 

Access to the Internet is widespread in developed countries. In the United States, for 
example, 4I° 0 of the population has access to the Internet [54]. 

While most consumers still access the Internet via low -speed dial-up modems connected 
to the switched telephone system, inexpensive broadband access is becoming available to 
a growing number of households via the cable netw orks. Cable service is available to 95% 
of American households [57], and cable modem service is available to a 20% subset [9]. 
While dial-up modems offer speeds of up to 56Kbps. cable modems offer practical speeds 
of up to about 3Mbps 1 , i.e. over 50 times faster. 

DSL (Digital Subscriber Line) [12,13], while o fieri ng similar speeds to cable modems but 
via the telephone system, is not yet widely used. ISDN (Integrated Services Digital Net- 
work), although widely used for corporate access, has had little consumer impact due to its 
high price and comparatively low performance. 

The deployment of third-generation (3G) cellular telephony within the next few years will 
bring practical mobile broadband speeds of 2Mbps [22]. 3G cellular uses WCDMA (wide- 
band code-division multiple access), a spread-spectrum technology. Satellite systems are 
arguably closer to offering even taster broadband Internet access [29.50,63]. 

The core of the Internet is made up of a number of independent high-speed fiber-optic net- 
works connected into NAPs (Network Access Points) or peered directly. These have until 
recently used single-wavelength TDM ( Time-Division Multiplexing) SONET (Synchro- 
nous Optical Network) transmission s\ stems which utilize about 1% of an optic fiber's 
capacin to yield a 2.5Gbps OC-48 channel^. Carriers are now beginning to deploy 
multi-wavelength DWDM (Dense Wavelength-Division Multiplexing) systems which 
yield up to 40 such channels per optic fiber, thus increasing network capacity significantly 
without requiring the laying of more fiber [10,40]. Internet architects are therefore now 
contemplating aggregate capacity in the terabit (T bps) range. 

The Internet uses the four-layer TCP/IP protocol suite I he application layer provides var- 
ious end-to-end application services, and is a client of the transport laver which provides 
end-to-end delivery-' services. T he transport layer in turn is a client of the network layer 
which provides packet routing. The network layer is a client of the link layer which encap- 
sulates specifics of the protocols and hardware of the actual communications links. 

The core Internet transport protocol, TCP (Transmission Control Protocol), provides a 
reliable end-to-end delivery service. The core Internet network protocol. II 5 (Internet Pro- 
tocol), provides an unreliable and connectionless packet routing service. IP ma\ lose or 



1 Although the cable supports 30 Mbps and the cable modem theoretical!) supports 10Mbps 

2 S( )Nf I channels have an ( K-n designation, u here ( R stands for Optical Carrier, and //gives the channel speed m 
units of about 5 2Mbps An OC-4 8 channel therefore has a speed of about 2 S( ibps 
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deliberate!) discard packets, and may deliver packets out of order, and it is the responsibil- 
ity of a higher laver to provide a reliable end-to-end service. 

With the proliferation of streaming media services on the Internet, support for multicast is 
spreading rapidly. Multicast is a form of broadcast with a specific set of recipients. It 
makes efficient use of network capacity because a packet traverses a network link once 
rather than once per recipient. It is particularly efficient if the recipients are connected to 
the Internet via an intrinsically broadcast medium such as cable or satellite. The @Home 
cable network has successfully enabled multicasting of streaming media services [38]. 

IP Multicast is an extension of IP, and so is unreliable. While this is often acceptable for 
time-critical data such as streaming video, it may not be acceptable for other shared data 
types. Significant effort is being expended to develop reliable multicast transport proto- 
cols on top of IP Multicast. Although several reliable multicast protocols are available and 
have been deployed [4 1 ,42,30,3 I ], the Internet standardization process is incomplete [32]. 

5.2 Netpages and Netpage Documents 

Netpages are the foundation on which a Netpage Network is built. They provide a 
paper-based user interface to published information and interactive services. 



Netpage Faqe Server 



object 



(1^1) 



N f -r r .*, r 



ot'ifcr 























doc mtjt \d 











Silverbrook Research Netpage System Overview 



draft vO 5 



Hach Netpage consists of a compact page layout maintained persistently hv a Netpaue 
Page Server. The page lay out refers to objects such as images, fonts and pieces of text, 
typically stored elsewhere on the Netpage Network. 

Netpages are organized into Netpage Documents. Both Netpages and Netpage Documents 
are assigned globally unique identifiers. 

Each Netpage Document has a set of document instances, each of which describes a 
printed instance of the document. Each Netpage in the Netpage Document has a corre- 
sponding set of page instances, each of which describes a printed instance of the page. 
Both page instances and document instances are assigned globally unique identifiers. 
They are also uniquely associated w ith the printer on which they are printed and the user 
who initiated the print request, if known. 

Each page instance maintains a set of user-supplied values for fields in the page layout. 
This ensures that user input is captured and stored independently for each page instance. 
The separation of page instances and Netpages is crucial for pages which contain input 
fields, i.e. forms. It is not crucial for pages devoid of input fields, but still useful because it 
supports independent auditing of each page instance. 

The physical page image includes encoded information which identifies the page instance 
and hence the Netpage to which it corresponds. It also includes encoded information 
which superimposes an addressable spatial grid over the page image, to allow pen actions 
performed relative to the page image to be correlated with the contents of the page layout. 

I he encoded information is normally printed in infrared-absorptive ink on any normal 
paper substrate which is infrared-reflective. Near-infrared wavelengths are invisible to the 
human eye but are easily sensed by a solid-state image sensor with an appropriate filter. 

1 he encoded information is picked up by an infrared-imaging pen and transmitted to the 
associated Netpage Printer. The pen is wireless and communicates with the Netpaue 
Printer using an encrypted radio frequency signal. 

T he encoded information is organized as a set of tags, each containing both the id of the 
page instance and the position of the tag. The tags tile the entire page image, and are suffi- 
ciently small and densely arranged that the pen can reliably image at least one taa even on 
a single click on the page. It is important that the pen recognize the page instance id and 
position on every interaction w ith the page, since the interaction is stateless. 

The tags are error-correct ably encoded to make them resilient to errors introduced bv dirt 
on the page or during the imaging process. 

Mem jet-based Netpage Printers are designed to print a tag grid using infrared (IK) ink. 
Printers not enabled for IR printing have the option to print tags using I R-absorptive black 
ink. although this restricts tags to otherwise empty areas of the page. Although such paues 
have more limited functionality than IR-printed pages, they are still classed as Netpaucs. 

5.3 The Netpage Network 

A Netpage Network consists of a distributed set of Netpage Publication Servers, Netpaue 
Page Servers, and Netpage Printers connected via an internet. In technological terms this 
document describes any Netpage Network. In business terms it usual Iv refers to the 
Netpage Network connected via /he Internet. 
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£Lt"n ,Cd ab0Ve -K ; : NCtpagC ,>agC SLTVCr main,ains Persistent information ahou. 
Netpage Documents. Netpages. and ,hoir printed instances, to allow pen operations on 
printed pages to be interpreted intelligently. on 

The Netpage Network includes any number orNetpagc Page Servers, each handling a sub- 

d which PaSeS , H eS t nbed ab ° VC ' Cadl ^ ,nSUinCe iS iden ' if «* b >' a globally- umque 

tse IS ,? C m ' I" ° f ,hC COrres P° ndi "g Printed page. The Netpage Printer 

scs « llb Id to retneve (he pagc , ayou( of (he ^ from a Nc ^ P 0 «t 

needs to interpret pen operations relative to the page. *" 

The Netpage Printer uses the internet Distributed Name System (DNS) to resolve i 
Netpage mstance ,d into a page instance maintained by a particular Netpage Page Server. 

The DNS is a protocol and a hierarchical system of name servers used to resolve internet 
dornatn names tnto resources. Planned enhancements to the DNS allow i, to be used to 
resolve more general Uniform Resource .dentifiers (UR.s). and in particular Un forn" 

" eTaTuRNs 1 ", URNS) H in,0 H rCS0U : Ce '° Ca,i ° nS ^ insta "« ' ds ™ 

lattd as URNs. allowmg the enhanced DNS to be used to resolve them in ,h„ ,h„„„„ „r 

Nelworr nda H d 'T ,IOn dCpl ° ymem of an enhance d DNS on the Internet die^a^ 
Network can deploy tts own system of enhanced name servers. 

A Netpage Publication Server is an internet server which publishes Netpage Documents to 
Netpage Printers. It is described in Section 6. documents to 

5.4 The Netpage Printer 

NeLaS P N 8 eK Pn r er * *■ Wh ' Ch prims Netpage documents. It is connected to a 

Netpage Network via an internet, ideally via a broadband connection. 

Apart from identity and security settings in non-volatile memorv. the Ne.paee Printer con- 
tan s no pers.sten, storage. As far as a user ,s concerned, //,, ne « twk ,£ \. omputcr , 6 ° 
Ne pages funcnon tnteractively across space and time with the help of the distributed 
Netpage Page Servers, independently of particular Netpage Printers. 

The Netpage Printer receives Netpage Documents from Netpage Publication Servers 

obi Jc^hirr ' S ,W ° Pam: PagC la ' V0L1,S " and ^ ««t and imase 

objects which populate the pages. Because of personalization, page layouts are typically 
specfic to a particular subscriber and so arc pointcas, to the subscriber s printer Zu nd 
,e ob c, 0 h )|hcr hand aro iypical|> . sharej u|(h Hthcr subscr P crs ■ 

niulliLast to all subscribers' printers. 

Poim^Tr,H |,U,1 ! iC ' : ' ,i0n ^ C ' AC ' ° P """ /L ' S ^ m -<'""" 1 ^"",e m content m,o 
pomi.as.s and mulncasts Alter receiving the pointcast ol a document's pa»o lavouts the 
printer knows which multicasts, if any, to listen to. ' 

Once the printer has received the entire document's page descriptions, i.e pa-e lavouts 
ami objects, it can print the document. ' ' "' 

I he printer rasteri/es and prints odd and even pa->es <imi,lu ,„.,„,. i, u.,,,. : , 
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one or more standard DSPs running in parallel. The duplexed print engines consist of cus- 
tom processors which expand, dither and print page images in real time, synchronized 
with the operation of the printheads in the imaging units 

I here are four major design variations embodied in the various Netpage Printer models; 

• form factor: pocket, portable, desktop, wall-mount or tabletop 

• printhead width. 4' 1 (photo), $V 2 " (portrait Letter) or 11" (landscape Letter) 

• paper source: cut sheet or print cartridge 

• Internet connection: wired or wireless 

The form factor variations yield five basic models, each with variants determined by print- 
head width (and hence printing speed), and paper source. Light planned models are 
defined in Table 1, and illustrated in Figure 2. 



Table 1. Netpage Printer models 



model 


form 
factor 


variant 


printhead 
width 


paper 
source 


Internet 
connect 


Microprinter 


pocket 


R 


4" 


cartridge 


wireless 


Travelprtnter 


portable 


R 


8VT 


cartridge 


wireless 


Deskprinter 


desktop 


R 


cartridge 


wired 
or 

wireless 


Wallprmter 


wall-mount 




cut sheet 


Pro 


11 " 


cut sheet 


Pro R 


cartridge 


Tablepnnter 


tabletop 


Pro 


cut sheet 


Pro R 


cartridge 



I he Deskprinter. Wallprmter and Tableprinter models can be factor -configured with var- 
ious network modules, allow ing both wired and wireless versions. The Microprinter and 
Travclprinter both use a cellular telephone module, with the promise of broadband speed 
within a few years. 

Hie Wallprmter models are ideal for unobtrusive installation in a home, while the Table- 
printer models might be preferred in an office environment. Note that the Tableprinter 
models are Wallprmter models factory -adapted for tabletop use via a stand. The 
Deskprinter, with its small footprint, is idea! for both home and office use. 

I he Microprinter prints norma! Netpages at quarter si/e. and provides full wireless 
Netpage Network access in a pocket device. 

I he paper roll cartridge contains both paper and ink. The paper is in the form of a contin- 
uous roll, cut on demand by the printer. The I I " print cartridge has a capacity of 1 000 Let- 
ter sheets. It also contains the glue supply for binding the sheets of a document together. 
The X' V' print cartridge has a capacity of 50 Letter sheets, or equivalentiv l 00 A5 sheets 
I he 4" print cartridge has a capacity of 36 6x4 photos, or 4 1 quarter-size Netpages. The 
and 4" print cartridges don't contain a glue supply because neither the Microprinter 
nor the I ra\ elpnnter includes a binding mechanism. 

The 4" printhead models print at 30 quarter-size pages per minute. The %v 2 " printhead 
models print at 60 pages per minute, or 30 duplex sheets per minute The I 1" printhead 
models print at 90 pages per minute, or 4 5 duplex sheets per minute. 
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5.5 The Netpage Pen 

I he Netpage Pen operates both as a normal marking ink pen and as a non-marking stylus. 
When either nib is in contact with a Netpage. the pen continuously monitors its move- 
ments relative to the page. The nib is attached to a pressure sensor. The pen pressure can 
be interpreted relative to a threshold to indicate whether the pen is "up" or "down". It can 
also be interpreted as a continuous value, for example when the pen is capturing a signa- 
ture, to allow the full dynamics of the signature to be verified. 

The pen determines the position of its nib on the Netpage by imamm*. in the infrared spec- 
trum, an area of the page in the vicinity of the nib. It decodes the nearest page id and posi- 
tion tag, and adjusts the position given by the tag to account for the distance between the 
area imaged and the actual nib, and the position of the tag in the imaged area. Although 
the position resolution of the tag may be low, because the tag density on the page is 
inversely proportional to the tag size, the adjusted position resolution is quite high, and 
easily exceeds the minimum 200 dpi resolution required for handwriting recognition [62], 

Pen actions relative to a Netpage consist of a series of strokes. A stroke consists of a 
sequence of time-stamped pen positions on the page, initiated by a pen-down event and 
completed by the subsequent pen-up event. A stroke is also tagged with the page id of the 
Netpage whenever the page id changes, i.e. just at the start of the stroke under normal cir- 
cumstances. 

The position tags on the Netpage contain various control bits. One of these instructs the 
pen to activate its "active area" LED. Thus a region on the page which corresponds to the 
active area of a button or hy perlink can be encoded to activate this LED, giving the user 
visual feedback that the button or hyperlink is active when the pen passes over it. Another 
control bit instructs the pen to capture continuous pen pressure readings and tag the stroke 
with these readings. Thus a region on the page which corresponds to a signature input area 
can be encoded to capture continuous pen pressure. 

Whenever the pen is within range of a printer with which it can communicate, the pen 
slowly flashes its ''online" LED When the pen fails to decode a stroke relative to the page, 
it momentarily activates its "error" LED. When the pen succeeds in decoding a stroke rel- 
ative to the page, it momentarily activates its "ok" LED. 

The pen also contains a pair of passive accelerometers mounted at right angles to each 
other in the plane normal to the pen's axis. The accelerometers respond to gravity and 
allow the pen to compute its tilt. This in turn helps it auto-focus its optics and compute the 
nib-to-tag displacement. If the stroke is being tagged w ith pen pressure readings, then it is 
also tagged with tilt readings. 

A sequence of captured strokes, whether tagged with pen pressure and tilt or not, is 
referred to as digital ink. Digital ink forms the basis for the digital exchange of draw ings 
and handwriting, for on-line recognition of handwriting [62], and for on-line verification 
of signatures. 

The pen is wireless and transmits digital ink to the Netpage Printer using a radio frequency 
signal. The digital ink data is encrypted for security and packet i/ed for efficient transmis- 
sion, but is alwa\s Hushed on a pen-up event to ensure timely handling in the printer. 

When the pen is out-of-range of a printer it buffers digital ink in internal memory, w hich 
has a capacity of more than 12 minutes of continuous handwriting. When the pen is once 
again within range of a printer, it transfers an\ buffered digital ink. 
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A pen can ho registered with any number of primers, but because all stale data resides in 
Netpages both on paper and on the network, it is largely immaterial which printer a pen is 
communicating with at any particular time. 



5.6 Netpage Interaction 



When the Netpage Printer receives a digital ink stroke from the pen, it retrieves the page 
layout of the Netpage identified in the stroke, to allow it to correctly interpret the stroke. 
The printer resolves, via the DNS, the address of the Netpage Page Server w hich holds the 
page layout, and then retrieves the page layout from the server. If the page was recently 
identified in an earlier stroke, then the printer may already have the address of the relevant 
Netpage Page Server in its cache. It may also have the page layout itself in its cache, in 
which case there may be no need to retrieve it. 

Once the printer has the page layout of the Netpage to w hich the pen stroke refers, it can 
interpret the stroke in relation to the layout and content of the page. This involves hit-test- 
ing the objects on the page to determine which objects the pen is interacting with, in much 
the same way that mouse movements and button presses are interpreted in a graphical user 
interface system. 

A "click" is a stroke where the distance between the pen down position and the subsequent 
pen up position is less than some small maximum. An object which is activated by a click 
requires a click to be activated, i.e. a longer stroke is ignored. The failure of a pen action, 
such as a "sloppy" click, to register is indicated bv the lack of response from the pen's 

-ok" li;d. 

There are two kinds of interactive objects on a Netpage: hy perlinks and form fields. 

When a hyperlink is activated, the printer sends a request to a handler somewhere on the 
network. The handler is identified by a URI. and the URI is resolved in the normal wa\ \ ia 
the DNS. There are three types of hyperlinks: general hyperlinks, form hyperlinks, and 
selection hyperlinks. A genera! hyperlink may implement a request for a linked document, 
or may simply signal a preference to a server. A form hyperlink submits the corresponding 
form to a form handler. A selection hyperlink submits the current selection to a selection 
handler. If the current selection contains a single-word piece of text, for example, the 
selection handler ma> return a single-page document giving the word's meaning within 
the context in which it appears, or a translation into a different language. L-ach hyperlink 
type is characterized by what information is submitted to the handler 

form fields come in four varieties: checkboxes, text areas, digital ink areas, and signature 
areas. A checkbox accepts a true or false \alue. An\ mark (a tick, a cross, a stroke, a till 
/ig/ag. etc.) captured in a checkbox area is assigned as a true value to the corresponding 
held. A text area accepts a text stung. An> digital ink captured in a text area is automati- 
cally converted to text via on-line handwriting recognition and the text is assigned to the 
corresponding held A digital ink area accepts raw digital ink. Any digital ink captured in 
a digital ink area is assigned to the corresponding field. A signature area accepts a hand- 
written signature Any digital ink captured in a signature area is automatically verified and 
the resulting signature token is assigned to the corresponding signature Held. Signature 
verification is discussed in more detail in Secimn X 
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"Ldiling" commands, such as strike-throughs indicating deletion, are also recognized in 
form fields. 



Table 2. Summary of pen interactions with a Netpage 



object 


type 


pen input 


action 


hyperlink 


general 


click 


submit action to handler via URI 


form 


click 


submit form to handler via URI 


selection 


click 


submit selection to handler via URI 


form field 


checkbox 


any mark 


set field value to true 


text area 


handwriting 


convert digital ink to text, 
assign text to field 


digital ink area 


digital ink 


assign digital ink to field 


signature area 


signature 


verify digital ink signature; 
assign signature token to field 


none 




circumscription 


convert digital ink to region; 
select object(s) in region 



Because the handwriting recognition algorithm works "on-line" (i.e. with access to the 
dynamics of the pen movement), rather than "off-line" (i.e. with access only to a bitmap of 
pen markings), it can recognize run-on discretely-written characters [62] with high accu- 
racy, w ithout a writer-dependent training phase. 



Digital ink. as already stated, consists of a sequence of strokes. Any stroke w hich starts in 
a particular object's active area is appended to that area's digital ink stream, ready for 
eventual interpretation. Any stroke not appended to an object's digital ink stream is 
appended to the remaining inactive area's digital ink stream. 

Digital ink captured in the inactive area is interpreted as a selection gesture. Anv circum- 
scription of one or more objects is interpreted as a selection of the circumscribed objects. 

The printer maintains a current selection for each pen. The selection contains the most 
recent object selected, resolved with reference to the page layout and content. The selec- 
tion can be attached to or pasted into another form, or in general be submitted to a selec- 
tion handler as described earlier The selection is cleared alter an inactivity time-out to 
ensure predictable behavior. 

Table 2 prov ides a summary of pen interactions w ith a Netpage. 

5.7 Forms 

As described in Section 5.2, user input on a physical Netpage is ultimately recorded per- 
sistently by a Netpage Page Server together with the corresponding page instance. To 
ensure efficient capture of user input, the printer accumulates input locally. To prevent 
update anomalies, however, the printer temporarily obtains exclusive access to the paue 
instance from the Netpage Page Server. The printer Hushes input back to the server and 
relinquishes exclusive access when the user initiates a non-local action on the page; after 
an inactivity time-out on the page; when the printer wishes to free up local storage con- 
sumed by the page, and on request from the server. 

When the printer submits a form to a form handler, it simply submits the document 
instance of the form I he form handler retrieves the Held values from the Netpage Pa^e 
Server at its leisure 
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A form can also act as a shared "blackboard" between the user and the form handler, i.e. 
the form handler can query the contents of the form fields maintained by the Netpa«c Paue 
Server without the user explicitly submitting the form. 

F-'or text areas, the raw digital ink is optionally also stored with the page instance on the 
Netpage Page Server. This allows the form handier to interrogate the raw digital ink 
should it suspect the original recognition of the handwriting. This might involve human 
intervention at the application level for forms which fail certain application-specific con- 
sistency checks. As an extension to this, the entire background area of a form can be des- 
ignated as a digital ink area. The form handler can then decide, on the basis of the 
presence of digital ink outside the explicit fields of the form, to route the form to a human 
operator, on the assumption that the user may have indicated amendments to the filled-in 
fields outside of those fields. 



f orm fields can optionally be tagged to indicate their meaning. Fields tagged in this way 
may include name and address fields, for example. This semantic tagging allows these 
fields to be automatically filled in whenever a "blank" form is requested by an identifiable 
user i.e. a user who has registered their identity with the s.vsiem and linked it to the iden- 
tity of their pen. 



5.8 Standard Features of Netpages 

fiach Netpage is printed with the Netpage logo at the bottom to indicate that it is a Netpage 
and therefore has interactive properties. The logo also acts as a "copy" button. In most 
cases pressing the logo produces a copy of the page. In the case of a form the button 
instead elicits a page giving the user the option to print the entire form document. And in 
the case of a secure document, such as a ticket or coupon, the button elicits an explanatory 
note or advertising page. 

1 he detauh single-page copy function is handled directly by the relevant Netpage Pane 
Server. Special copy functions are handled by linking the logo button to other URIs. 

Once a Netpage form has been submitted, it is marked as submitted by the Netpage Paue 
Server and cannot be submitted again. An attempt to do so elicits a status report indicating: 
when it was submitted. A copy of the form can still be made, altered, and re-submitted. 



5.9 The Help System 



I he Netpage Printer has a single button labelled "help". W hen pressed it elicits a single 
page of information 1 his information includes the follow in- 

• status of printer connect ion 

• status of printer consumables 

• top-level help menu 

• document function menu 

• top-le\el Netpage Network directory 

1 1. . \ . I I ; . . i 
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• print a clean copy of a form 

• print the status of a document 

A document function is initiated by simply pressing the button and then touching any page 
ol the document. The status of a document indicates who published it and when, to whom 
it was delivered, and to whom and when it was subsequently submitted as a form. 

The Netpage Network directory allows the user to navigate the hierarchy of publications 
and services on the network. As an alternative, the user can call the Netpage Network 
"900" number "yellow pages" and speak to a human operator. The operator can locate the 
desired document and route it to the user^s printer. Depending on the document type, the 
publisher or the user pays the small "yellow pages" service fee. 

The help page is obviously unavailable if the printer is unable to print. In this case the 
"error" light is lit and the user can request remote diagnosis over the network. 
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6 Personalized Publication 
Model 



In the following discussion, news is used as a canonical publication example to illustrate 
personalization mechanisms in the Netpage System. Although news is often used in the 
limited sense of newspaper and newsmagazine news, the intended scope is wider. 

In the Netpage System, the editorial content and the advertising content of a news publica- 
tion are personalized using different mechanisms. The editorial content is personalized 
according to the reader's explicitly stated and implicitly captured interest profile. The 
advertising content is personalized according to the reader's locality and demographic. 



A subscriber can draw on two kinds of news sources: those that deliver news publications, 
and those that deliver news streams While news publications are ausreaated and edited b\ 
the publisher, news streams are aggregated either by a news publisher or by a specialized 
news aggregator. News publications typically correspond to traditional newspapers and 
newsmagazines, while news streams can be many and varied: a "raw" news feed from a 
news service, a cartoon strip, a freelance writer's column, a friend's bulletin board, or the 
reader's own e-mail. 

1 he Netpage Publication Server supports the publication of edited news publications as 
w ell as the aggregation of multiple new s streams. By handling the aggregation and hence 
the formatting of news streams selected directly by the reader, the server is able to place 
advertising on pages over which it otherwise has no editorial control. 

1 he subscriber builds a daily new spaper by selecting one or more contributing news pub- 
lications, and creating a personalized version of each. The resulting daily editions are 
printed and bound together into a single newspaper. The various members of a household 
typically express their different interests and tastes by selecting different daily publica- 
tions and then customizing them. 

l or each publication, the reader optionally selects specific sections. Some sections appear 
daily, while others appear weekly. The daily sections available from The New York Times 
online, for example, include "Page One Plus", "National", "International", "Opinion". 

"business", "Arts Living". " I echnologv ". and "Sports". 'I he set of available sections is 
ohv iolisK specific to a publication, as is the default subset 

I he reader extends the daiK newspaper bv creating custom sections, eaefi one draw in lz on 
an\ number ot news streams Custom sections might be created lor e-mail and friends' 
announcements ("Personal"), or for monitoring news feeds for specific topics ("Alerts" or 
"Clippings"). 

for each section, the reader optionallv specifies its si/e, either qualitativ el\ (e.g. short, 
medium, or long), or numerically (i.e. as a limit on its number of pages), and the desired 

proportion of advertising either qualitativ el\ (cv hi<.*h normal low m.mi<m or pumvt- 




6.1 



Editorial Personalization 
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An article may also bo written (or edited) in different versions to match the expected 
sophistication of the reader, for example to provide children's and adults 1 versions. The 
appropriate version is selected according to the reader's age. The reader can specif\ a 
"reading age" w hich takes precedence over their biological age. 

The articles which make up each section are selected and prioritized by the editors, and 
each is assigned a useful lifetime. By default they are delivered to all relevant subscribers, 
in priority order, subject to space constraints in the subscribers' editions. 

In sections where it is appropriate, the reader may optionally enable collaborative filter- 
ing. This is tli en applied to articles which have a sufficiently long lifetime. Hach article 
which qualifies for collaborative filtering is printed with rating buttons at the end of the 
article. The buttons can provide an easy choice (e.g. "liked" and "disliked*), making it 
more likely that readers w ill bother to rate the article. 

Articles with high priorities and short lifetimes are therefore effectively considered essen- 
tial reading by the editors and are delivered to most relevant subscribers. 

The reader optionally specifies a serendipity factor, either qualitatively (e.g. do or don't 
surprise me), or numerically. A high serendipity factor lowers the threshold used for 
matching during collaborative filtering. A high factor makes it more likely that the corre- 
sponding section w ill be filled to the reader's specified capacity. A different serendipitv 
factor can be specified for different days of the week. 

The reader also optionally specifies topics of particular interest within a section, and this 
modifies the priorities assigned by the editors. 

The speed of the reader's Internet connection affects the quality at which images can be 
delivered. I he reader optionally specifies a preference for fewer images or smaller images 
or both. If the number or size of images is not reduced, then images may be delivered at 
lower quality (i.e. at lower resolution or with greater compression). 

At a global level, the reader specifies how quantities, dates, times and monetary values are 
localized This involves specifying w hether units are imperial or metric, a local timezone 
and time format, and a local currency, and whether the localization consist of in situ trans- 
lation or annotation. These preferences are derived from the reader's locality by default. 

To reduce reading difficulties caused by poor eyesight, the reader optionally specifies a 
global preference for a larger presentation. Both text and images are scaled accordingly, 
and less information is accommodated on each page. 

The language in which a news publication is published, and its corresponding text encod- 
ing, is a property of the publication and not a preference expressed by the user. However, 
the Netpage Network may provide automatic translation services in various guises. 



The personalization of the editorial content directly affects the advertising content, 
because advertising is typically placed to exploit the editorial context. Travel ads. for 
example, are more likely to appear in a travel section than elsewhere. The value of the edi- 
torial content to an advertiser (and therefore to the publisher) lies in its ability to attract 
large numbers of readers with the right demographics. 



6.2 



Advertising Localization and Targeting 
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Iflective advertising is placed on the basis of locality and dcmoitraphics. Locality deter- 
mines proximity to particular services, retailers etc., and particular interests and concerns 
associated with the local community and environment. Demographics determine -eneral 
interests and preoccupations as well as likely spending patterns. 

A news publisher's most profitable product is advertising "space-, a multi-dimensional 
entity determined by the publication's geographic coverage, the size of its readership its 
readership demographics, and the page area available for advertising. 

In the Netpage System, the Netpage Publication Server computes the approximate 
multi-dimens.onal size of a publication 's saleable advertising space on a per-section basis 
taking into account the publication's geographic coverage, the section's readership the 
size of each reader's section edition, each reader's advertising proportion, and each 
reader s demographic. 

In comparison with other media, the Netpage System allows the advertising space to be 
defined .n greater detail, and allows smaller pieces of it to be sold separately It therefore 
allows it to be sold at closer to its true value. 

For example, the same advertising "slot" can be sold in varying proportions to several 
advertisers, with individual readers' pages randomly receiving the advertisement of one 
advertiser or another, overall preserving the proportion of space sold to each advertiser. 

The Netpage System allows advertising to be linked directly to detailed product informa- 
tion and online purchasing. It therefore raises the intrinsic value of the advertising space. 

Because personalization and localization are handled automatically by Netpage Publica- 
tion Servers, an advertising aggregator can provide arbitrarily broad coverage of both 
geography and demographics. The subsequent disaggregation is efficient because it is 
automatic. 1 his makes it more cost-effective for publishers to deal with advertising a«»re- 
gators than to directly capture advertising. liven though the advertising aggregator is'tak- 
ing a proportion of advertising revenue, publishers may find the change profit-neutral 
because of the greater efficiency of aggregation. The advertising aggregator acts as an 
intermediary between advertisers and publishers, and mav place the same advertisement in 
multiple publications. 

It is worth noting that ad placement in a Netpage publication can be more complex than ad 
placement ,n the publication's traditional counterpart, because the publication's advertis- 
ing space more complex. While ignoring the full complexities of negotiations between 
advertisers, advertising aggregators and publishers, j, ,, clear that the Netpafe Svstcm 
should idealK provide some automated support for these negotiations, includm- support 
I'"- automated auctions of advertising space. Automation ,s particularK desirable for the 
Placemen, ol advertisements which generate small amounts ot income, i.e. small or I.Hrlv 
localized advertisements. ~ ' 

Once placemen, has been negotiated, the aggregator captures and edits the advertisement 
and records it on a Netpage Ad Server. Correspondingly the publisher records the ad 
placemen, on the relevant Netpage Publication Server When the Nctpaec Publication 
Server lavs out each user's personalized publication, it nicks the relevant adv.-rti 



■m t'r 



Stlverbrook Research Netpage System Overview 



draft vO 5 



6.3 User Profiles 



I he personalization of news and other publications relies on an assortment of user-specific 
profile in format ion: 

• publication custom ixations 

• collaborative filtering vectors 

• contact details 

• presentation preferences 

The customization of a publication is typically publication-specific, and so the customiza- 
tion information is maintained by the relevant Netpage Publication Server. 

A collaborative filtering vector consists of the user's ratings of a number of news items. 
As described in Section 4, it is used to correlate different users 1 interests for the purposes 
ot making recommendations. Although there are benefits to maintaining a single collabo- 
rative filtering vector independently of any particular publication, there are two reasons 
why it is more practical to maintain a separate vector for each publication: there is likelv 
to be more overlap between the vectors of subscribers to the same publication than 
betw een those of subscribers to different publications; and a publication is likely to want 
to present its users' collaborative filtering vectors as part of the value of its brand, not to 
be found elsewhere. Collaborative filtering vectors are therefore also maintained bv the 
relevant Netpage Publication Server. 

Contact details, including name, street address. ZIP Code, state, country, telephone num- 
bers, etc.. are b> their nature global and are maintained by a Netpage Registration Server. 

Presentation preferences, including those for quantities, dates and times discussed in Sec- 
tion 6.1. are likewise global and maintained in the same way. 

I he localization of advertising relies on the locality indicated in the user s contact details, 
while the targeting of advertising relies on personal information such as date of birth, gen- 
der, marital status, income, profession, education, etc., or qualitative derivatives such as 
age range and income raime. 

I or those users who choose to reveal personal information for advertising purposes, the 
information is maintained by the relevant Netpage Registration Server. In the absence of 
such information, advertising can be targeted on the basis of the demouraphic associated 
with the user's ZIP or ZIP t 4 Code. 

f ach user, pen. printer, publisher and publication is assigned its own globallv unique iden- 
tifier, and the Netpage Registration Server maintains the relationships between them 

Kach user may be authorized to use any number of printers, and each printer mav allow 
an\ number of users to use it. bach user has a single default printer, to which periodical 
publications are delivered. 1 he server keeps track of which publishers a user has autho- 
rized print to the user's default printer. 

fach user may have several pens but a pen is specific to a single user. If a user is autho- 
rized to use a particular printer, then that printer recognizes any of the user's pens. 

These relationships are illustrated in I mure V 
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Figure 3. User registration relationships 

The pen identifier is used, in the form of a URN. to locate the corresponding user profile 
maintained by a particular Netpage Registration Server, via the DNS in the usual way. 

A Web terminal can be authorized to print on a particular Netpage Printer, allowing Web 
pages and Netpage documents encountered during browsing to be conveniently printed on 
the nearest Netpage Printer. 



Intelligent Page Layout 

The Netpage Publication Server automatically lays out the pa«es of each user's personal- 
ized publication on a section-by-section basis. Since most advertisements are in the form 
of pre-formatted rectangles, they are placed on the page before the ed, tonal content. 

I he advertising ratio for a section can be achieved w ith wildly vamrm advertising ratios 
on individual pages within the section, and the ad layout algorithm exploits this The algo- 
rithm attempts to co-locate closely tied editorial and advertising content, e » ads for roof- 



mire on 



"immaterial placed specifically with the publication because of a special fe; 
do-it-yourself rooting repairs. 

The editorial content selected for the user, i.e. text with associated .mages and graphics, is 
then laid out according to various aesthetic rules 



I lie entire process, including the selection of ads and the selection or editorial content 
mus ! h "' lkT:iUv: oncc th ^' ^>out has ouixcrycd. to attempt to more cW.seh achieve the 
users stated section s!/e preference I he section si/e preference can. houever. he 
matched lifl ./ivn^-mcr lime, allowing significant da\ -to-da\ variations. 

Document Format 



Once the document is laid out, it is encoded for efficient distribution and persistent storage 
on the Netpag v ' ^ 
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A text object contains fully-formatted text represented in the Extensible Markup Lan- 
guage (XMI.) [68] using the Extensible Stylesheet Language (XSL) [69]. XSL provides 
precise control over text formatting independently of the region into which the text is 
being set. which in this case is being provided by the layout. The text object contains 
embedded language codes to enable automatic translation, and embedded hyphenation 
hints to aid with paragraph formatting. 

An image object encodes an image in the JPEG 2000 wavelet-based compressed ima»e 
format [34]. The original DCT-based JPEG algorithm introduces negligible visual loss^at 
compression ratios below 10:1 [65]. JPEG 2000 is planned to achieve The same quality at 
compression ratios 30% higher, i.e. at about 13: 1 [35]. 

A graphic object encodes a 2D graphic in Scalable Vector Graphics (SVG) [7 1] format. 

I he lay out itself consists of a series of placed image and graphic objects, linked textflow 
objects through which text objects flow, hyperlinks and input fields as described in Sec- 
tion 5.6, and watermark regions. These layout objects are summarized in Table 3. The lay- 
out uses a compact format suitable for efficient distribution and storage. 

1 he layout is tagged with the version of the text-setting algorithm used by the Netpage 
Publication Server when the layout was first created, allowing the Netpage Printer u> 
exactK reproduce the physical layout intended by the server. 

Because Netpage Printer software is automatically upgraded over the Netpaae Network, it 
is feasible for Netpage Printers to contain every' version of the text-settiim algorithm. 

Table 3. Netpage layout objects 



layout 
object 


attribute 


format of 
linked object 


image 


position 




image object URI 


JPEG 2000 


graphic 
textflow 


position 




graphic object URI 


SVG 


te <tflow id 




region 3 




optional text object URI 


XML/XSL 


hyperlink 
field 

watermark 


type 




region 3 
handler URI 
type 

meaning 




region 3 
region 3 





a arbitrary multi-edged shape defined with spline paths 



6.6 Document Distribution 

As described above, for purposes of efficient distribution and persistent storage on the 
Netpage Netuork. a user-specific paue la\out is separated from the shared objects to 
u hich it refers. 
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Whet, a subscribed publication is ready to be distributed, the Netpaue Publication Server 
allocates, with the help of the Netpage Id Server, a globally unique id for each pane, page 
instance, document, and document instance. " ' ^ 

The server computes a set of optimized subsets of the shared content and creates a multi- 
cast channel lor each subset, and then tags each user-specific layout with the names of the 
multicast channels which will carry the shared content used by that layout The server then 
pomtcasts each user's layouts to that user, and when the pointcastinu is complete multi- 
casts the shared content on the specified channels. After receiving its pointcast each 
Netpage Printer subscribes to the multicast channels specified in the paue layouts l)urin<> 
the multieasts, each printer extracts from the multicast streams those objects referenced by 
its page layouts. 

Once a printer has received all the objects to which its page layouts refer the printer 
re-creates the fully-populated layout and then rasterizes and prints it. 

The server also delivers each page layout to the relevant Netpage Page Server, which m .,v 
* V C ° ! , 0caJcd " 4V,th ti,c Net P a S e Publication Server, or may be located elsewhere on the 
network. I hus the page layouts are persistently archived as Netpaues. It is the responsibil- 
ity of the Netpage Publication Server to preserve the shared objects referenced by the 
Netpages, to ensure that they are really persistent. It may choose to archive these shared 
objects elsewhere on the network a. any time. The object URls embedded in the Netpa«cs 
allow the objects to move. 

Under normal circumstances, the printer prints panes faster than thev can be delivered 
Assummg a quarter of each page is covered with images, the average pane has a s,/e of 
less than 400KB . The printer can therefore hold in excess of 100 such pages in its internal 
64MB memory, allowing for temporary buffers etc. The printer prints at a rate of one pa«e 
per second. I his is equivalent to 400KB or about 3Mbit of pace data per second vehicles 
similar to the highest expected rate of page data delivery over a broadband network. 

Even under abnormal circumstances, such as when the printer runs out of paper it is likelv 
that the user will be able to replenish the paper supply before the printer's 1 00-p a - o inter- 
nal storage capacity is exhausted. 

However, it the printer's internal memory does fill up. then the printer will be unable to 
make use of a multicast when it first occurs. The Netpage Publication Server therefore 
allows printers to submit requests for re-nn.lticasts. When a critical number of requests ,s 
received or a timeout occurs, the server re-multicasts the corresponding shared objects. 

| 'nee a document is printed, a Netpage Printer can produce an exact duplicate at anv time 
h> retrieving its page layouts from the relevant Netpage Cage Serve, and removing the 
objects to which thev refer from the network 



6.7 On-Demand Documents 



When a Netpage document is requested ad hoc. it is personalized and delivered in much 

the same wav as a periodical However, since there is no shared content deliv,-.-. i 
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When a non-Nctpage document is requested ad hoc, it is not personalized, and it is deliv- 
ered via a designated Netpage Formatting Server which reformats it as a Netpage docu- 
ment. A Netpage Formatting Server is a special instance of a Netpage Publication Server 
The Netpage Formatting Server has knowledge of myriad Internet document formats, 
including old favorites such as Adobe's Portable Document Format (PDF) [4], and Hyper- 
text Markup Language (HTML) [70]. In the case of HTML, it makes use of the higher res- 
olution of the printed page to present Web pages in a two-column format, with a table of 
contents and an index of links. By default it automatically includes all Web pages directly 
linked to the requested page. The user can tune this behavior via a preference. 

The Netpage Formatting Server makes standard Netpage behavior, including interactivity 
and persistence, available on any Internet document, no matter what its origin and format. 
It hides know ledge of different document formats from both the Netpage Printer and the 
Netpage Page Server. 
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7 Security 

7.1 Cryptography 

Cryptography is used to protect sensitive information, both in storage and in transit, and to 
authenticate parties to a transaction. There are two classes of cry ptography in widespread 
use: secret-key cryptography and public-key cryptography. The Netpage Network uses 
both classes of cry ptography. 

Secret-key cry ptography, also referred to as symmetric cryptography, uses the same key to 
encry pt and decrypt a message. Two parties wishing to exchange messages must first 
arrange to securely exchange the secret key. 

Public-key cryptography, also referred to as asymmetric cryptograph), uses two encryp- 
tion keys. The two keys are mathematically related in such a way that any message 
encrypted using one key can only be decry pted using the other key. One of these keys is 
then published, while the other is kept private. The public key is used to encry pt any mes- 
sage intended for the holder of the private key. Once encrypted using the public key, a 
message can only be decrypted using the private key. Thus two parties can securely 
exchange messages without first having to exchange a secret key. To ensure that the pri- 
vate key is secure, it is normal for the holder of the private key to generate the key pair. 

Public-key cry ptography can be used to create a digital signature. If the holder of the pri- 
vate key creates a know n hash of a message and then encry pts the hash using the private 
key. tti en anyone can verify that the encrypted hash constitutes the "signature" of the 
holder of the private key with respect to that particular message, simply by decrypting the 
encrypted hash using the public key and verifying the hash against the message. If the sig- 
nature is appended to the message, then the recipient of the message can verity both that 
the message is genuine and that it has not been altered in transit. 

To make public-key cryptography work, there has to be a way to distribute public keys 
which prevents impersonation. This is normally done using certificates and certificate 
authorities. A certificate authority is a trusted third part> which authenticates the connec- 
tion between a public key and someone's identity'. The cert i tic ate authority- verities the 
person's identity by examining identity documents etc., and then creates and signs a digi- 
tal certificate containing the person's identity details and public key. Anyone who trusts 
the certificate authority can use the public key in the certificate with a high degree of cer- 
tainty that it is genuine. They just have to verify that the certificate has indeed been signed 
by the certificate authoritv. whose public kev is well-known 

In most transaction environments, puhlic-kev crvptographv is onl\ used !o create digital 
signatures and to securely exchange secret session keys. Secret-key crvptographv is used 
for all other purposes. 

In the follow ing discussion, w hen reference is made to the svo/f/v tr ansm ission of informa- 
tion between a Netpage Printer and a server, what actually happens is that the printer 
obtains the server's certificate, authenticates it with reference to the certificate authoritv. 
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7,2 Netpage Printer Security 

Kach Netpage Printer is assigned a pair of unique identifiers at time of manufacture which 
are stored in read-only memory in the printer and in the Netpage Registration Server data- 
base The first id is public and uniquely identifies the printer on the Netpage Network. The 
second id is secret and is used when the printer is first registered oti the network. 

When the printer connects to the Netpage Network for the first time after installation, it 
creates a signature public/private key pair. It transmits the secret id and the public kev 
securely to the Netpage Registration Server. The server compares the secret id against the 
printer's secret id recorded in its database, and accepts the registration if the ids match. It 
then creates and signs a certificate containing the printer's public id and public signature 
key, and stores the certificate in the registration database. 

The Netpage Registration Server acts as a certificate authority for Netpage Printers, since 
it has access to secret information allowing it to verify printer identity. 

When a user subscribes to a publication, a record is created in the Netpage Registration 
Server database authorizing the publisher to print the publication to the user's default 
printer. Every document sent to a printer is addressed to a particular user and is signed by 
the publisher using the publisher's private signature key. The printer verifies, via the regis- 
tration database, that the publisher is authorized to print the publication on the specified 
user's default printer and that the printer is the user's default printer. The printer verifies 
the signature using the publisher's public key, obtained from the publisher's certificate 
stored in the registration database. 

1 he Netpage Registration Server accepts requests to add printing authorizations to the 
database, so long as those requests are initiated via a pen registered to the printer. 

1 he user can authorize a Web terminal to print on a printer. This is useful if the user has a 
Web terminal in the home which is used to locate documents on the Web for printing. The 
one-time authorization proceeds as follows: the user prints a Web terminal authorization 
form. The Netpage Registration Server generates a short-lifetime one-time-use id for the 
Web terminal which is printed on the form, together w ith the URI of the printer The Web 
terminal is used to navigate to a Netpage Registration Server registration site, where the 
one-time- use id is entered, as well as the URI of the printer. The Web terminal generates a 
signature public/private key pair. The server allocates a terminal id for the Web terminal, 
creates and signs a certificate containing the terminal id and the public key, and stores the 
certificate in the registration database. The URI of the printer, the Web terminal's terminal 
id. and the private signature key are stored locally on the Web terminal. 

Whenever the Web terminal wishes to print on the printer, it sends the printer's designated 
Netpage formatting Server a request containing the URI of the document to be printed, 
together with the terminal id, signed with the Web terminal s private signature kev. On 
receipt of the request and before acting on it, the server verifies, via the registration server, 
thai the terminal is authorized to print on the specified printer, and verities the signature 
using the terminal's public key. obtained from the terminal s certificate stored in the reuK- 
tration database. 

I he user can print a list of current printing authorizations at anv time, and revoke an\ 
which are beim> abused. 
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7.3 Netpage Pen Security 

Hach Netpage Pen is assigned a unique identifier at time of manufacture which is stored in 
read-only memory in the pen and in the Netpage Registration Server database. The id 
uniquely identifies the pen on the Netpage Network. 

A Netpage Pen can know a number of Netpage Printers, and a printer can know a number 
of pens. A pen communicates with a printer via a radio frequency signal whenever it is 
within range of the printer. Once a pen and printer are registered, they regularly exchange 
session keys. Whenever the pen transmits digital ink to the printer, the digital ink is always 
encry pted using the appropriate session key. Digital ink is never transmitted in the clear. 

A pen stores a session key for every printer is knows, indexed by printer id, and a printer 
stores a session key for every pen it knows, indexed by pen id. Both have a large but finite 
storage capacity for session keys, and will forget a session key on a least-recently-used 
basis if necessary. 

When a pen comes within range of a printer, the pen and printer discover whether they 
know each other, if they don't know each other, then the printer determines whether it is 
supposed to know the pen, i.e. because the pen belongs to a user who is registered to use 
the printer. If the printer is meant to know the pen but doesn't, then it initiates the auto- 
matic pen registration procedure. If the printer isn't meant to know the pen, then it agrees 
with the pen to ignore it until the pen is placed in a charging cup, at which time it initiates 
the registration procedure. 

In addition to its public id, the pen contains a secret id and a secret key-exchange key, both 
intended for one-time use. These are also recorded in the Netpage Registration Server 
database at time of manufacture. During registration, the printer obtains the secret id from 
the pen. Because it is transmitted in the clear, it may be intercepted by someone listening 
in. The printer transmits the id securely to the Netpage Registration Server, which 
responds securely w ith the matching key-exchange key. together with a new t\ -uenerated 
secret id and key-exchange key. The printer generates a session key for the pen and trans- 
mits it to the pen encrypted using the one-time-use key-exchange key. It also securely 
transmits the new secret id and key-exchange key to the pen, which saves them for the 
next registration procedure. They now match the pen's record in the Netpage Registration 
Ser\ er database 

It the secret id transmitted in the clear from the pen to the printer is intercepted and used to 
retrieve the secret key-exchange key from the Netpage Registration Server before the 
printer queries the server, then the server rejects the printer's querv because the secret id is 
out-ol-date. Thus the printer knows that the pen ha^ been compromised, and recommends 
that it be returned for repair 

When a previouslv unregistered pen is first registered, it is of limited use until it is linked 
to a user. A registered but "un-owned" pen is only allowed to be used to request and fill in 
Netpage user and pen registration forms [ >). i.e. to register a new user to which the new 
pen is automat icalK linked, or to add a new pen to an existing user. 

I he pen uses secret-kev rather than public-ke\ encryption because of hardware perfor- 
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7.4 



Secure Documents 



The Netpage System supports the delivery of secure documents such as tickets and cou- 
pons. The Netpage Printer includes a facility to print watermarks, but will only do so on 
request from publishers who are suitably authorized. The publisher indicates its authority 
to print watermarks in its certificate, which the printer is able to authenticate. 

The "watermark" printing process uses an alternative dither matrix in specified "water- 
mark" regions of the page. Back-to-back pages contain mirror-image watermark regions 
which coincide when printed. The dither matrices used in odd and even pages' watermark 
regions are designed to produce an interference efTect when the regions are viewed 
together - i.e. when looking through the printed sheet. 

The effect is similar to a watermark in that it is not visible when looking at only one side 
of the page, and is lost when the page is copied by normal means. 

As described in Section 5.8, pages of secure documents cannot be copied using the built-in 
Netpage copy mechanism. This extends to copying Netpages on Netpage-aware photo- 
copiers. 

Secure documents are typically generated as part of e-commerce transactions. They can 
therefore include the user's photograph which was captured when the user registered bio- 
metric information with the Netpage Registration Server, as described in Section 8. 

When presented with a secure Netpage document, the recipient can verify its authenticity 
by requesting its status in the usual way. The unique id of a secure document is only valid 
lor the lifetime of the document, and secure document ids are allocated non-contiguouslv 
to prevent their prediction by opportunistic forgers. A secure document verification pen 
can be developed with built-in feedback on verification failure, to support easy 
point-of-presentation document vent] cat ion. 

Clearly neither the watermark nor the user's photograph are secure in a cryptographic 
sense. They simply provide a significant obstacle to casual forgery. Online document veri- 
fication, particularly udng a verification pen, provides an added level of security where it 
is needed, but is still not entirely immune to forgeries. 



In the Netpage System, forms submitted b\ users are delivered reliably to forms handlers 
and are persistently archived on Netpage Page Servers, it is therefore impossible for recip- 
ients to repudiate delivery. 

I -commerce pav merits made through the svstem, as described in Section 8. are also 
impossible for the payee to repudiate. 



7.5 



Non-Repudiation 
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8 Electronic Commerce Model 

8.1 Secure Electronic Transaction (SET) 

The Netpage System uses the Secure Electronic Transaction (SE I ) [43] system as its pay- 
ment system model. Although SET is not yet widely supported, it is comprehensive and 
elegant and will probably become dominant in the near future. 

SET, having been developed by MasterCard and Visa, is organized around payment cards, 
and this is reflected in the terminology. However, much of the system is independent of 
the type of accounts being used. 

In SET, cardholders and merchants register with a certificate authority and are issued with 
certificates containing their public signature keys. The certificate authority verifies a card- 
holder's registration details with the card issuer as appropriate, and verifies a merchant's 
registration details with the acquirer as appropriate. Cardholders and merchants store their 
respective private signature keys securely on their computers. During the payment pro- 
cess, these certificates are used to mutually authenticate a merchant and cardholder, and to 
authenticate them both to the payment gateway. 

SET has not yet been adopted widely, partly because cardholder maintenance of key s and 
certificates is considered burdensome. Interim solutions which maintain cardholder keys 
and certificates on a server and give the cardholder access via a password have met with 
some success [ 14]. 

8.2 SET Payments 

In the Netpage System the Netpage Registration Server acts as a proxy for the Netpage 
user { i.e. the cardholder) in SET pay ment transactions. 

The Netpage System uses biometrics to authenticate the user and authorize SET payments. 
Because the system is pen-based, the biometric used is the user's on-line signature, con- 
sisting of time-varying pen position, tilt and pressure. A fingerprint biometric can also be 
used by designing a fingerprint sensor into the pen, although at a higher cost. The type of 
biometric used only affects the capture of the biometric, not the authorization aspects of 
the system. 

I he first step to being able to make SI I payment is to register the user's biometric with 
the Netpage Registration Ser\er. I his is done in a controlled en\ ironment. for example a 
hank, where the biometric can he captured at the same time as the user's identil\ is \eri- 
iied I he biometric is captured and stored in the registration database, linked to the user's 
record and to the record of a particular Netpage Pen. The user's photograph is also option- 
ally captured and linked to the record. I he SET cardholder registration process is com- 
pleted, and the resulting private signature key and certificate are stored in the database 

I he user's payment card in format ion is also stored, giving the Netpage Registration 
Server enough information to act as the user's proxy in any SET payment transaction. 
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8.3 



Micro-Payments 



The Netpage Network includes a mechanism for micro-payments, to allow the user to he 
conveniently charged for printing low-cost documents on demand and for copying copy- 
right documents, and possibly also to allow the user to be reimbursed for expenses 
incurred in printing advertising material. The latter depends on the level of subsidy 
already provided to the user. 

When the user registers for e-commerce, a network account is established which aggre- 
gates micro-payments. The user receives a statement on a regular basis, and can settle any 
outstanding debit balance using the standard payment mechanism. 

The network account can be extended to aggregate subscription fees for periodicals, which 
would also otherwise be presented to the user in the form of individual statements. 



Whenever a transaction originates through a Netpage form, the form handler has sufficient 
information, in the shape of the form's unique document instance id, to maintain transac- 
tion-specific state information. However, a transaction may also originate through a 
non-form page such as a printed catalog page, implying, for example, the existenee^of a 
virtual "shopping cart". In this case the relevant transaction state information is tied, indi- 
rect ly. to the unique id of the user. 

I he Netpage Registration Server maintains an anonymous relationship between a user and 
a transaction handler via a uniquely numbered transaction, as illustrated in Figure 4. 
Whenever the user activates a hyperlink tagged with the "transaction" attribute, the 
Netpage Printer asks the Netpage Registration Server to translate the associated handler 
id. together with the pen id. into a transaction id. The transaction id is then submitted to 
the hyperlink transaction handler. For efficiency, the printer caches transaction ids 

The transaction handler maintains state information indexed by transaction id. It is able to 

retrieve user-specific state information w ithout explicit knowledge of the user. 



8.4 



Transactions 





Figure 4. User transaction 
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9 Applications 

I he Netpage Network has the potential to subsume a wide var iety of applications in both 
traditional and electronic media. This section sketches the following possibilities: 

• personalized subscriptions to newspapers, magazines and comics 

• subscriptions to freelance columns and bulletin boards 

• on-demand newspapers, magazines and comics 

• on-demand flyers and product brochures 

• on-demand books 

• television infotainment printing 

• e-commerce purchasing from online and traditional catalogs 

• delivery of statement and invoices, with online payment 

• delivery of secure document (tickets, coupons and licenses) 

• pe r fe c t c opy i n g w i t h co py right m ic ro- pay m e n t s 

• mail replacement 

• delivery of greeting cards 

• form printing, fill-in. and submission 

• delivery of e-mail and facsimile 

• active business cards 

• on-demand document delivery on corporate intranets 

• provi sion of government services 

• Web browsing, searching and printing 

• photo album creation 

• persistent searchable note taking 

• exam taking and homework 

• computer printing 

Manx of t he so applications are described in detail in [ H 

9.1 Personalized Subscriptions 

The strength of the Netpage Network lies in automatically delivering subscribed periodi- 
cals, at a print quality equalling or exceeding that of their traditional counterparts, with 
editorial content personalized to individual interests, advertising content localized and tar- 
geted to individual localities and demographics, and advertising directly linked to detailed 
product information and product purchasing. 

9.1.1 Newspapers and Magazines 

I he Netpage Network offers a new delivery mechanism to the S24 billion newspaper and 
newsmagazine market which is both more cost-effective than centralized printing and dis- 
tribution, and allows more fine-grained targeting of advertising. 

I he simplest form ot "news personalization involves selecting a news publication and 
choosing which daily and weekly sections to receive. The simplest form of advertising 
personalization is tuned to the demographics associated with the subscriber's ZIP or 
/IP* 4 code Kven with these entry-level forms of personalization, the Netpage Network 
otters a compelling distribution model for news 
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Users receive paper publications with the usability of their traditional counterparts, but 
with interactive properties. At the press of an on-page button, a user can print an article 
giving the background to a news story, print a personalized product brochure, or add a 
product to the virtual shopping basket. 

Any magazine normally printed on lightweight paper stock is equallv well-suited to distri- 
bution on the Netpage Network. However, since Netpage Printers don't carry heavier 
paper stocks and don't provide wrap-around binding, the Netpage Network is less 
well-suited to the distribution of so-called "glossy^ magazines. 

9.1.2 Freelance Columns and Bulletin Boards 

Users can choose to subscribe to individual freelance columns, cartoons, etc. These can be 
integrated into a user's daily news document, or printed individually, freelancers can 
choose to receive micro-payments from their subscribers, freeing them from maintaining 
their own subscriber databases. The Netpage Network provides mechanisms for handling 
micro-payments. 

Users can also subscribe to the "bulletin boards 1 ' of friends: collections of news, 
announcements, pictures etc., which work much like freelance columns. 

9.2 On-Demand Publications 

The Netpage Network can deliver, on demand, current and back issues of periodicals nor- 
mally delivered on subscription, including newspapers, magazines, and comics. To main- 
tain the interactivity of all Netpages ever printed, the Netpage Network keeps all 
published content online at all times. Unlike the Web, where hyperlinks become unreliable 
over time, content on the Netpage Network never expires. 

9.2.1 Flyers and Product Brochures 

The Netpage Network makes high-quality flyers and product brochures instantly avail- 
able, linked to advertisements and entries in printed catalogs. 

Brochures arc always up-to-date, and link to e-commeree. e-mail, and automatic telephone 
call-back. Brochure links can provide "click-through" fees to linking documents, and sub- 
sidized printing to users. 

Active Netpage advertisements can appear in printed paper publications sold or delivered 
through traditional outlets, including newspapers and magazines sold on news stands, 
posters appearing in public [Maces (including in buses, trams and ta\u ). and books sold in 
bookshops Buvin^ guides and travel guide books are particuIarK lilted to contamin- 
ant i\e Netpage advertising:. 

9.2.2 Books 

Users can obtain the latest best-sellers or rare "out-of-print" (a soon-to-be-obsoleted term) 
editions on demand, printed in column format with a text size chosen by the user. A typi- 
cal .*U)0-page paperback His on as little as 40 sheets of Letter paper Slip-on covers are 

■iv -lilaLU tor rn^n r >- , 
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Colorful children's books reproduce immaculately. When they've been loved to death, 
I hey can be printed again, and again. 

Children's coloring-in books and puzzles are available just w hen thev're needed on a rainy 
day. 

9.2.3 Television Infotainment 

Users watching infotainment programs on television can print the associated informa- 
tional material on their Netpage Printer by pressing the "print" button on their remote con- 
trol at the appropriate time. The material may be a report on a new medical procedure, the 
plans for a do-it-yourself bookshelf, or a list of top investment opportunities. It may also 
be a subscription form for the print publication the program is promoting. 

The television set and/or remote control is suitably modified or augmented to route the 
print request to the Netpage Printer. The television or remote control may, for example, 
include a radio transmitter or transceiver which allows it to communicate with the 
Netpage Printer. When the print button is pressed, a message is sent to the Netpage Printer 
which contains the identity of the television channel. The telev ision channel id. date and 
time together uniquely identify the Netpage document associated with the program at that 
time. The printer constructs a URN based on the television channel id and determines the 
address of the television channel's document server v ia the DNS in the usual way. It then 
retrieves the document from the document server based on date and time. 

A potentially more flexible approach embeds the URN of the document in the telev ision 
signal itself, for example as a closed caption. In analog television signals there is plenty of 
room for information in the vertical blanking interval (VBI). The advantage of leverauinu 
support for closed captioning, in particular, is twofold. Closed caption decoders are now 

standard components in normal-sized telev ision sets, and closed captioning is fullv sup- 
ported in digital television standards that are now being deployed. Other VBI-based 
schemes require custom decoders and may not survive the transition to digital television. 

By routing the television set's video output to the Netpage Printer, and locating a closed 
caption decoder in the printer, modification of the television set can be avoided entirelv. 

9.3 E-COMMERCE 

9.3.1 Online Purchases 

The Netpage Network supports a similar level of online purchasing as the Web. but in 
pa per- based medium which presents like a high-quality printed catalog. 

A user can navigate the retailer's online Netpage catalog, printing catalog paizes as they 're 
needed and adding items to a virtual shopping trolley. The contents of the shopping cart 
can be listed at any time, and items can be struck from the list at the stroke of the pen. 
Pressing a "proceed to checkout" button at any time elicits a completed order form just 
waiting for the user's signature. The payment card account number is securely shown in 
the usual I 234 56** **** *789 format. The user's signature authorizes the payment 

9.3.2 Catalog Purchases 

Rather than buying from an online Netpage catalog, the user can select items from a tradi- 
tional l\ -printed catalog which contains active Netpage links 
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9.3.3 Statements and Invoices 

Statements and invoices can be securely and auditably delivered, and can be automatically 
tilled in with the user's default payment details without the sender knowing those details' 

I he user's signature can authorize the payment as normal. 

9.3.4 Secure Documents 

Retailers can securely issue tickets and coupons over the Netpage Network, printed with 
difTicult-to- forge watermarks. 

Agencies of various kinds can issue licenses printed w ith watermarks and the user's own 
photograph. 

As described earlier, recipients can verify the authenticity of secure documents using a 
standard Netpage Printer or a special verification pen. 

9.3.5 Copyright Copying 

Any printed version of a Netpage document becomes an easy means to printing another 
perfect copy. When a cop\ is made, the Netpage Network can automatically transfer a 
micro-payment from the copier to the copyright holder. 

Trivial copyright fees are universally respected but seldom paid because of the inconve- 
nience. The Netpage Network offers micro-payment convenience and the quality of an 
original copy. 

9.3.6 Authenticated Signing and Exchange of Legal Documents 

Netpage supports authenticated delivery and printing of documents, and automatical l\ 
captures and verities signatures entered in signature fields. It can therefore support authen- 
ticated delivery and signing of legal documents. 

9.3.7 Retail Signature Capture 

Point-of-purchase signature capture is a common requirement when the customer uses a 
payment card or charge card. Some retailers are now adopting digitizing tablets with 
LCDs to capture signatures, both to reduce paperwork and potentially to implement signa- 
ture verification in the future. A Netpage paper roll which is printed with the details of 
each transaction and then captures the customer's signature provides an alternative. The 
customer benefits from the ease of using an ink pen on paper, while the retailer benefits 
from online signature capture and verification. 

9.4 Communication 

9.4.1 Mail 

I he Netpage Network, once widely subscribed, can be used to deliver numerous instances 
of regular mail-outs, particularly statements and invoices as discussed in tectum () ^ ; ^ 
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9.4.2 Greeting Cards 

A user can select a greeting card from an online catalog, add a handwritten message, and 
dispatch it via the Netpage Network. Cards can be addressed to other Netpage users, and 
to normal postal addresses. In the latter case the card is printed at the service center closest 
to the recipient, automatically placed in an envelope, and mailed through the local mail 
system. 



Netpage users can choose to receive cards from anyone, or only from authorized friends. 

9.4.3 Forms 

f orms of all kinds can be printed on the Netpage Printer, tilled in by hand, and submitted 
directly over the Netpage Network. Submission is secure and cannot be repudiated. 

Handwriting is automatically recognized by the system, t he digital ink of the handwriting 
is attached to the form in case a human clerk needs to re-interpret the handwriting. Auto" 
matic "handwriting bots" on the network can assist with the recognition task, automati- 
cally giving the user semi-intelligent feedback to elicit disambiguation. 

Any interactive Netpage "application* 1 , including e-commerce and e-mail, uses forms of 
various kinds. 



9.4.4 E-Mail 

f.-mail forms can be printed on demand and filled in b> hand. The handwritten name of 
the recipient is converted to facilitate delivery, but the rest of the message is delivered as 
digital ink, just as the user intended. If the recipient is computer-based rather than 
Netpage-based, all of the handwriting can be automatically converted, with the digital ink 

sent as an attachment (since it may contain hand-drawn diagrams etc). 

Netpage users can choose to receive e-mail from an v one, or onlv from authorized friends. 

I-aeh user maintains a list of contacts which allows e-mail to be addressed by name. Users 
never deal with Netpage user ids directly. If the specified name is ambiguous, the svstem 
prints a list of choices. The use of nicknames helps to avoid ambiguitv. 



9.4.5 Business Cards 

I.aeh user's business card can act as a convenient one-time e-mail authorization token. It is 
encoded as a Netpage and contains a button which, when pressed by the card recipient, 
adds the card owner to the recipient's contact list and authorizes the recipient to send the 
card owner e-mai I. 



I he business card can also contain buttons which link to company information, personal 
information, and even product information if the user is a salesperson. 

1 lie Netpage Network provides a bureau service which allows users to order active 
Netpage business cards. 



9.4.6 Facsimile 

f acsimile forms can be printed on demand and filled in by hand. The handwritten tele- 
phone number is converted to facilitate delivery, hut the rest of the message is delivered as 
bitmapped digital ink, just as the user intended. 
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9.4.7 Conferencing 



Geographically distributed participants can communicate graphical ideas via a shared 
Netpage "whiteboard" while engaged in a telephone conference. Every time a participant 
adds a modification or a new diagram to the whiteboard the other participants are given a 
new copy of the page. New pages can be started at any time, but older pages or older ver- 
sions of pages can still be marked up and re-distributed. Each participant" ends up with a 
full history of the interaction. Multiple colors and line styles can be chosen from a palette. 

Each printed page is timestamped and marked with the name of the author of the latest 
change. The latest change can be shown in a different style to the older content, to indicate 
what is new. For example, if the whiteboard is monochrome, then the latest change can be 
shown in color, or if the whiteboard is in color, then the latest change can be shown in bold 
strokes. 



9.4.8 Collaborative Document Markup 



Geographically distributed participants can mark up a shared Netpage document while 
engaged m a telephone conference. Every time a participant adds a modification or a new 
diagram to the document the other participants can be given a new copy of the page. A 
participant chooses to share their contribution with the other participants by pressing the 
< update > button which appears on every page. Older versions of pages can be marked up 
and re-distributed at any time. Each participant ends up with a full history of the interac- 
tion. Each participant's contribution can be shown in a different color, so that contribu- 
tions can be distinguished. A key printed on each page or an initial status page indicates 
which color corresponds to which participant. 

Each printed page is timestamped and marked with the name of the author of the latest 
change. Beyond color-coding each user's contribution, the latest change can be shown in a 
different style to older markup, to indicate what is new. for example, the latest chanae can 
be shown in bold strokes. 

A session chairperson creates the markup session and links a Netpage document to it. The 
chairperson then e-mails session invitations to potential participants. Each invitation con- 
tains an accept button, which, when pressed, adds the inv itee to the session. The invita- 
tion also typically contains the date and time that the session is intended to start, presented 
in the local time of the invitee, as well as the name of the document and the purpose of the 
markup. Ihe chairperson receives e-mail every' time an invitee accepts the invitation. The 
chairperson can distribute a markup session control page to the participants at any time, 
either before or during the actual session. The control page contains the session details, 
lists the session participants and shows their assigned markup colors, and contains a link 
to the document being marked up whereby the participant can obtain a copy. It also con- 
tains an • update - button which is used by the participant dunm- the session to distribute 
their contribution to the other participants. 

Ihe chairperson's session control page also contains start session ■ and ■ end session - 
buttons No markup is possible before the session is started or alter the session is ended. 

Ihe chairperson can designate non-contributory participants - i.e. ones who can monitor 

the markup m- ^ mim < ^-^r m • - 1 ..^ o>. r .... .i 
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A non-Neipage document can be marked up by first turning it into a Netpage document 
i.e. by printing it to a Netpage Printer in Netpage mode (as described in Section 9.6.5). 



9.4.9 Postage Stamps 



I ostage stamps can now be purchased over the Internet and printed on an Inkjet printer 
Netpage provides an even more compelling purchase and delivery mechanism for postage 
stamps because of its seamless payment mechanisms (for explicit payments and implicit 
m icro-payments) and its support for secure documents. 

As Nctpages. Netpage-delivered postage stamps can contain both attractive graphics and 
invisible machine-readable information such as source and destination codes allow 
mail systems to track mail items and automatically route them. 



ving 



9.4.10 Voicemail 



I he Netpage System can provide access to nominated voicemail mailboxes maintained by 
a telephone system. The help page can contain a <voicemail> button which prints a list of 
voicema.l messages received by the user of the pen. Each message shows the date and 
time of rece.pt. the name of the caller (determined by looking up™ address book usim- 
the phone number provided by caller ID), and the status of the message - i e whether it has 
been played back yet. The user can organize the message list into a hierarchy of folders as 
described in Sect.on 6 in [3]. A message can be printed, in which case the system attempts 
u, convert the message speech into text. The message can be played back through an 
optional speaker ,n the Netpage Printer. The message can also be plaved back through the 
user s mobile phone, either by the Netpage Printer controlling the mobile phone via its 
short-range radio interface (i.e Bluetooth), or by the Netpage Printer instructing the tele- 
phone system to call the mobile phone and automatically plav back the specified message 



9 4.11 Interactive Plans 



nteract.ve plans can be printed as Netpages. The plans can be linked to a database and can 
be marked up with a Netpage Pen. Multiple people can collaborate on a single paper plan 
and small changes can he made to a plan on the shop floor, without requirm- access to i 
computer or an operator with computer skills. 

If the plan is of a three-dimensional object, then different views of the object can be 
selected using a 3D view control, as described in Section 9 in [3]. 

The plan might be for a shoe design, and might show a dimensioned view of the shoe as 
uell as a perspective view from a selected position. New dimensions can be entered in 
numeric fields and saved to the same plan file or to a new plan file. 



9.5 Corporate and Government 

9.5.1 Corporate Intranets 



An organization can use a private intranet-based Netpage Network to implement a docu- 
ment repository and elf mentis distribute documents on demand. 
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9.5.2 Government Services 



Government can provide access to services via the Netpage Network. The network can 
obviate the need to visit government offices to obtain forms and submit forms, and the net- 
work can be used to efficiently deliver the results of submissions. 



9.6 Personal 



9.6.1 Web Browsing, Searching and Printing 

Users can browse the World Wide Web via their Netpage Printer using paper and pen as 
the user interface. Netpage forms can provide emulation of HTML forms. Only dvnamic 
media objects may fail to print meaningfully. " ' 

A Netpage Printer can be the ideal output device for documents encountered while brows- 
ing the Web, whether the browsing is terminal-based or Netpage-based. An increasin« 
number of print-ready documents are being published on the Web. 

Lntire Web sites can be compactly formatted for print, since a printed page has a much 
greater mtormat.on-bearing capacity than a computer screen, and perused^in a more lei- 
surely fashion. The Netpage Formatting Server automatically adds a table of contents and 
an index of referenced pages. 

Links on Web pages remain active when printed on Netpages. Clicking on a link causes 
the linked Web page to be printed. 

Web pages and search results are printed on both sides of the paper, allowing one sheet of 
paper to hold the equivalent of ten SVGA screens of information. Thus the user sees 200 
links on paper, compared with only 20 on a computer screen. 

9.6.2 Photo Album 

A Netpage Printer can be enhanced w ith an infrared data connection f IrDA) to allow it n> 
accept images uselessly from a digital camera. Alternatively digital cameras with USB 
ports can be directly connected. Images can be automatically archived on the Netpa»e 
Network, and individual photos, both regular and poster-sized, can be printed at photo 
quality on the printer. 

Smart knout software can assist with the interactive creation of photo album pages. rcad\ 
tor insertion in a cumulative famiK album 

9.6.3 Notebook 

A pre-printed Netpage notebook can be used to capture handwritten notes in a persistent 
fashion. Notes are captured on the network and are optionally timestamped and signed In 
a certificate authority to provide undeniable proof of priority for example with respect to a 
patent application. Since the handwriting is recognized and converted to text for indexirw 
purposes, the notes can also be searched by kevword, date. etc. 
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An examination paper can be delivered at a specific time to a distributed collection of stu- 
dents. The students can then be required to answer the questions on the paper within the 
allotted time. Their answers can be automatically captured at the end of the allotted time 
using the "blackboard" form model, and in many cases automatically graded. Once a»ain. 
unrecognized input can be routed to a human operator. 

Multiple choice questions can be automatically graded. 

Homework in general can be distributed and captured in a similar way. 

9.6.5 Computer Printing 

A Netpage Printer can be the ideal output device for a personal computer or workstation, 
whether directly-connected or on a local-area netw ork. The speed of the Netpage Printer 
and the quality of its output make it attractive to existing computer users. 

The Netpage Printers have a USB port as a standard feature. 

A Netpage Printer can be used to print output from a computer system both as normal 
pages and as Netpages. When printed as Netpages, the pages are implicitly stored persis- 
tently on the Netpage Network, and the printed Netpages therefore have much of the inter- 
activity of Netpages which originate on the netw ork. 

9.6.6 Netpage Photocopying 

A Netpage-aware photocopier can be used to turn normal pages into Netpages in a similar 
way. When a normal (non-Netpage) page is copied, a digital version of the page is stored 
persistently on a Netpage Page Server, and the page copied is printed as a Netpage which 
is linked in the usual way to its digital counterpart. The digital version of the page ma> 
consist purely of an image of the page, or it may include text obtained via optical character 
recognition, thus allow ing text-based searching. 

When a Netpage is copied, a perfect digital copy is produced, subject to the usual Netpaue 
secunt) considerations a.s discussed in Section 7.4. 

9.6.7 Games of Chance 

Many gambling games can be delivered through Netpage. With instant lotto, just print the 
form, mark the appropriate numbers, and press the submit button. Win or lose, the 
Netpage response is instant. 

With a group lottery or Keno, again just print the form, mark the appropriate numbers, and 
press the submit button. Wait for the announcement of the w inning numbers, for example 
on television. Winners are also not i tied by Netpage. 

9.6.8 Multi-Player Gaming 

Multi-player games can be standard games such as chess. The players take turns marking 
moves on a Netpage showing the status of the game After each move the player presses 
the submit button. The update is automatically printed on the other player's Netpage 
Printer. The opposing player can of course also be a game-playing program running on the 
Netpage Network. 

An exploratory multi-player game (such as Ultima Online) can be presented as a Netpaue. 
The Netpage shows the current view, the collection of items currently held, a map of the 
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area explored so tar, and the chatter of nearby players. The form provides a text input field 
for chatting with other players, and command fields for navigating the game world. The 
submit button submits any chat text and navigation commands and produces an updated 
page. Previously printed pages form part of an evolving map of the game world, and pro- 
vide the basis for hyperjumping to different locations in the game world under the appro- 
priate circumstances. 



9.7 Consumer Electronics 

9.7.1 Netpage Digital Camera 

A Netpage Printer built into a digital camera is used both as a Netpage Printer and as a 
photo printer. A digital camera containing a Netpage Printer can provide the option of 
printing its photos as Netpages, thus providing persistent storage of digital negatives on 
the Netpage Network. Each printed photo allows a pristine copy of itself to be produced at 
any time, even if the original print is severely degraded. 

The digital negative also records the date, time and location the photo was taken, and the 
settings of the camera at the time. It also records any artistic efTect applied to the photo 
when it was printed [ I J. so that the actual output can be reproduced. 

The camera user can route photos to particular personal albums on the Netpage network. 
The user can grant other Netpage users (such as family members) access to particular 
albums. People can thus easily share photos without having to get together and swap ncii- 
atives or prints. This may be particularly convenient when the camera user is travelling. 
I he camera user may also choose to defer printing of photos while travelling, and instead 
retrieve and print the photos on returning home, possibly in the form of printed pages suit- 
able for inclusion in a normal photo album (see Section 9.6.2). 



9.8 Advertising 



9.8.1 Reverse-side Advertising 

In many highly interactive Netpage applications, much of the information is supplied to 
the user one printed sheet at a time, since this is the minimum unit of Netpage output 
Although a printed sheet has the capacity to hold two printed Netpages, i.e. one on each 
side of the sheet, there is in many cases insufficient information to fill two complete 
Netpages. When a sheet is printed with free space on the reverse side, unobtrusive adver- 
tising can be inserted. Although the immediate impact of the advertising is reduced 
because the advertising doesn't interrupt the user, the advertisement has persistence., and 
can be encountered some time after the interactive session if" the User turns over the sheet 
u h lie tidv ing up or archiv ing. 

I he Netpage system detects sheets which are to be printed on one side only, and automati- 
cal l\ prints selected advertising material on the reverse side. 

Although each user may have the ability to disable reverse-side printing of advert isirm. the 
Netpaire s\\tem is likelv to reward umwx f. w- ,vp! |n- . - . i 
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9.9 Product Information and Registration 

9.9.1 Product Information 

I he packaging of a product can be printed as one or more Netpages containing buttons 
which link to various kinds of information about the product. A milk carton might contain 
buttons linked to information about the manufacturer, the product, the product in detail, 
recipes which involve the product, and advertising promotions involving the manufacturer 
or product, such as competitions. A music compact disc might contain buttons which link 
to information about the band and about the songs on the disc itself. A consumer product 
such as a television might contain a button which links to usage instructions. 

In each case the consumer presses the button with a Netpage pen and receives the linked 
information on the Netpage Printer. Information is always delivered in the language of the 
user. This is particularly useful in the case of consumer electronics products which are 
sold in international markets. 

Products can indicate both their instance identity (serial number) and class identity (uni- 
versal product code). Consumers can thereby obtain instance-specific information from 
the Netpage Network, such as known problems, recalls, use-by-date, etc., as well as 
class-specific information, such as generic product information. 

9.9.2 Product Registration 

f urther to Section 9.9. 1, a product can also contain a <register> button which, when 
pressed with a Netpage Pen, registers the Netpage user as the owner of the product. The 
user's contact information, which is already recorded on the Netpage System, is automati- 
cally transmitted to the product manufacturer who can record it in the their customer data- 
base. The registration process automatically adds the manufacturer to the user's e-mail 
contact list, thus allowing the manufacturer to send the user e-mail relevant to the product, 
Mich as related special offers, recall notices, etc. If the manufacturer abuses their e-mail 
priveleges. the user can bar them in the usual way. 

9.9.3 Product Information via Universal Product Code Bar Code 

Many of the advantages of printing or affixing Netpages on the surfaces of products (as 
described in Section 9.9.1 above) can be gained by simply enhancing the Netpage Pen to 
decode the universal product code (UPC) bar codes which universally adorn products. The 
pen transmits the UPC to the Netpage Printer, tagged as a UPC rather than as digital ink. 
I he printer constructs a U PC-specific URI and locates the corresponding resource on the 
Netpage Network via the DNS in the usual way. 

I he request thus ultimately elicits printed product information. The product page can also 
include the facility to enter the serial number of the device and register the user's ow ner- 
ship of it via a <register> button, with all the advantages described in Section 9.9.2 above. 

Product manufacturers can thus gain the benefits of Netpage linking for their entire 
installed base of products without making alterations to the products themselves. 

9.9.4 Product Information via Help Button 

A physical help button can be installed in consumer devices such as televisions, kitchen 
appliances, etc.. The help button is coupled to a radio transceiver in the device which com- 
municates an information request to the nearest Netpage Printer. The request can consist 



?5 November 1999 



Silverbrook Research 



Netpage System Overview 



draft vO 5 



of just the universal product code (UPC) of the device, or can contain instance-specific 
information such as the serial number of the device as well. 

I he information request is handled by the Netpage Printer in the same way as described in 
Section 9.9.1 or Section 9.9.3 above, and ultimately elicits printed product information. 

9.9.5 Context-Specific Product Help 

Further to Section 9.9. 1 , the entire surface of a product can be printed as a Netpage. Press- 
ing on any part of the surface with a Netpage Pen can then elicit printed Netpage output 
which provides product-specific help. The help is cither specific to the area pressed, or 
relates to the product as a whole. Thus the user of the product has instant access to helpful 
information about specific features of a product as well as the product as a whole. Each 
feature-specific help page is linked to the entire product manual. 

9.9.6 Product Ownership Tracking 

Further to Section 9.9.2 and Section 9.9.5, the entire surface of a product can be printed as 
a Netpage. Pressing on any pan of the surface with a Netpage Pen can then elicit printed 
Netpage output which describes the product and its current ownership. 

After the product is purchased, pressing on any part of the surface automatically registers 
the product in the name of the owner of the Netpage Pen. 

Anyone can determine the ownership of a product offered for sale simply by pressing on 
any part of its surface with a Netpage Pen. 

Ownership can only be registered by a new owner if the current owner has relinquished 
ownership by signing the the "selT portion of the product's status page. This places the 
product in an "un-owned" state. 

Product information and ownership is maintained either by the product manufacturer, as a 
service to its customers, or by a profit-oriented third party. A manufacturer-maintained 
product registration database has many advantages (see Section 9.9.2 above), but has the 
disadvantage that it may be abused for advertising purposes. A third-part\ -maintained 
product registration database has the disadvantage that it may incur a usage fee. 

I he shipping computer system of a product manufacturer can automatically transfer own- 
ership of products from the manufacturer to the distributor or retailer, and so on down 
through the pre-sale chain. The retail computer system of the retailer can automatical!) 
mark each sold item as free, or transfer ownership direct !> to the holder of the pavment 
card used to pay for the product. I he customer can also use a Netpage Pen at the point of 
sale to register- immediate o\\ ner^hip of the product 

Iraditiona! clearing-houses for stolen goods, such as pawn shops, can be required by law 
to check the ownership of all products presented to them. Since a Netpage-encoded prod- 
uct has an invisible encoding on most or all of its surface, it is difficult for a thief to 
remove it or even tell if it has been successfully removed. Conversely, it is incumbent on a 
potential buyer of a product to ensure that a clean reading can be obtained from its surface 
m> rli.it IN ounrf'Tip ran fw mM> ,^, r ( M- » a ■■ ' ■ i- ■ ' 
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9.10 Geographic Information 



9.10.1 Route Planning 



An automobile navigation system typically determines the location of the car via GPS, 
shows a local map to the driver on a color display, assist the driver in planning a route, and 
gives the driver verbal instructions on following a route, cued to the changing location of 
the car. The Netpage System provides a convenient way for a driver to specify a destina- 
tion to the navigation system's route planner. The user can press on a location on a 
Netpage street map and then press the <destination> button on the navigation system. The 
navigation system automatically programs the map location as the destination. The user 
also can press the <location> button on an advertisement or on someone's business card 
and then press the <destination> button on the navigation system. This yields a potentially 
more accurate location than one derived from a map. 

The same technique allows a location to be specified as the destination to any route plan- 
ning system, for example one provided by the Netpage System itself. A Netpage route 
planning system accepts a destination location in the same way. It can assume that the 
starting location is the location of the printer through which the user is interacting, but 
allows the user to explicitly specify a different starting location, as well as to modify the 
default route and specify route constraints in the usual way supported by route-planninii 
systems. 



9.10.2 Globe 



I he surface of a globe, i.e. a sphere representing the earth, can be printed as a Netpage. A 
user can then interact with the globe via the Netpage Pen. The user can request informa- 
tion about geographical features, cities, countries etc.; can request more detailed maps; 
can request the distance between two locations; can request a route including a number of 
locations; etc. In each case the Netpage Printer responds with the printed information. 

In the case of detailed maps, successively more detailed maps can be requested, culminat- 
ing m street maps where available. Many different map views are possible, e.g. showing 
topography, vegetation, annual rainfall, seasonal temperature, population, malaria inci- 
dence, etc. Multiple views can also be combined. 

The user can select a number of points of the globe simply by clicking on them in succes- 
sion with the pen. The user can also select a geographic region by circumscribing it with 
the pen. Subsequent operations, selected by pressing Netpage buttons printed on the base 
of the globe or on a handy command card, act on the selected points or the selected region. 
Different command cards can be provided for users of different sophistication, e.g. chil- 
dren and adults. 

Ideally the globe rolls freely on its base, rather than rotating on an axis, so that any area on 
its surface can be rotated into view. A cupped base may be inset with ball bearings, for 
example. The surface of the globe is coated so that extended interaction with the base, as 
well as with the Netpage Pen, does not degrade the Netpage surface encoding. 

I he globe can provide audio output as an alternative to printed output. This may be partic- 
ularly appropriate for interactive guessing games. The globe can ask the user to select the 
country which has a certain set of attributes or contains a certain geographic feature, and 
can rate the choice or give a hint once the user makes a choice. Guessing uames can in 
general involve an> number of players, or just one player and the globe. 
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10 Business Models 



10.1 System Principles 

The Netpage Network leverages the open technology and extensive infrastructure of the 
Internet. The widespread acceptance and growth of the Netpage Network is predicated on 
open competition rather than monopolistic practices. 

However, to provide an incentive to early investors, semi-exclusive licenses to Mem- 
jet-based Netpage Printer designs will be offered, as well as licenses to manufacture paper 
and ink consumables. 




Figure 5. Open structure of Internet-based Netpage Network 

I he network supports any number of independent participants, some of which have com- 
plementary roles, and some of which compete. The open structure of the network is illus- 
trated in figure 5. Content- re I a ted participants include the following: 

• news sources 

• publishers 

• news aggregators 

• freelance artists, writers, cartoonists 

• direct mailers 
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Infrastructure-related participants include the following: 

• server suppliers 

• network storage providers 

• communications carriers 

• Internet service providers (ISPs) 

• printer dealers 

• printer installation and servicing companies 

• ink and paper consumables dealers 

• consumables delivery companies 

Technology-related participants include the following: 

• research and development 

• chip makers (printheads, controllers, QA) 

• printer manufacturers 

• ink and paper consumables manufacturers 

I he strength of the network lies in the fact that publication and delivery are completely 
decoupled. This allows the delivery infrastructure to grow independently of the participa- 
tion of publishers. 

10.2 Bootstrapping 

Because consumers are unlikeK to be motivated to acquire a Netpage Printer until a vari- 
ety of publications and services are available, and because publishers will wait for an 
installed base before participating, the key to bootstrapping the network is to bundle the 
printer with a publication or service subscription, and possibly trimming profit margins in 
the growth stage. 

I here are several ways the manufacturing cost of the Netpage Printer, assumed to be well 
below SI 00, can be subsidized. Printer-based distribution can eliminate existing distribu- 
tion costs, offsetting the printer cost. The printer can provide a new mechanism for deliv- 
ering advertising, with advertising profits offsetting the printer cost. And the printer cost 
can be built into the subscription fee for a publication or service. 

I he cost of printing and delivering a newspaper normally exceeds the price of subscrip- 
tion [27,5 ) ]. The real profit lies in the advertising. The cost of the Netpage Printer is easily 
exceeded by one years' cost savings, allow ing a Netpage subscription, including a "free" 
printer, to be priced lower than a traditional subscription. A Netpage subscription o (Ye red 
to a customer already on the network would be priced correspondingly lower still. 

It the publication or service delivered via the Netpage Printer is sufficiently lucrative, then 
the publisher or provider may be able to subsidize not only the printer itself, but also its 
running costs. This can include Internet access, paper and ink consumables, and servicing. 
Demographics- in formed advertising may fall into this category. The more information 
customers reveal about themselves, the greater the value of the advertising to the advertis- 
ers, and so the greater the lev el of subsidization that can take place 

Parly investors who subsidize the installation of Netpage Printers mav be able to recover 
the investment and turn a profit merely by charging other publishers an access fee to the 
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printers they "own", perhaps for an interim period after installation, accordinu to a net- 
work-wide agreement. They may also be able to earn commissions on click-throughs and 
e-commerce transactions originating on pages printed on "their" printers. 

Similar approaches are already emerging in the general internet market. In the "FreePC" 
and related models [37J, personal computers are bundled with Internet access and the 
whole package is fully or partially subsidized by advertising and e-commerce. 

Most content-related participants in the Netpage Network, and even Internet service pro- 
viders, can benefit from directly investing in Netpage Printer deployment. 

10.3 Maturity 

Many of the bundling approaches are likely to remain applicable once the network 
becomes widely subscribed. It is possible that the bundling of the appliance (i e the 
Netpage Printer) with the service (be it Internet access or a publication subscription) will 
remain the dominant means of distributing the appliance, as it is in the cellular telephone 
mar.;et, and as an increasing number ot companies, including IBM f l 8], are besinnin° to 
believe it should be in the personal computer market. " & 

10.3.1 News Publishers and News Aggregators 

News publishers with strong brands are likely to be able leverage those brands on the 
Netpage Network. They have an incentive to do so quickly to prevent newcomers from 
tilling the vacuum and capturing the attendant advertising revenue. 

News publishers also have an incentive to migrate to the Netpatie Network because it 
allows them to offer the more fine-grained targeting that advertisers are increasingly 
demanding, and which they are increasingly seeking elsewhere. 

News publishers who create content rather than simplv au»re«atino other sources have a 
significant advantage, since they offer both unique content and auditorial voice. Users 
are more likely to choose a single news publication whose content and editorial orienta- 
tion they find useful, than specifying to a news aggregator how to glue together a number 
ot disparate news sources. And any sufficiently strong news publication brand is unlikely 
to make its content available to an aggregator, since the aggregator will be taking a pro- 
portion of advertising and e-commerce revenue. 

The Netpage Network, like the Web. offers lower barriers to entry than traditional publish- 
ing media, and this naturally stimulates greater divers, tv. However, the ueoaraphic inde- 
pendence ot the network, coupled with built-in mechanisms for* localization of 
publications, allows international, national and regional news publications to more easiN 
compete m local news markets. 

The strength ofa traditional local new s publication lies partly in its local news content and 
partK m its local retail advertising and classified advertising content. Aggregation of clas- 
sified advertising is already happening on the Web, and the Netpaue Network will make 
the same thing possible for local retail advertising. Local news publications are therefore 
likely to he excluded from the direct capture o'' local lK^t; • „ . ,„ 
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10.3.2 Advertising Aggregators 



The Netpage Network promises to be the most effective advertising medium ever con- 
ceived. It combines the editorial and print quality of traditional publications with arbi- 
trary finely targeted advertising, and provides a direct link between advertising product 
information, and purchasing. 

Because personalization and localization are handled automatically by Netpage Publica- 
tion Servers, an advertising aggregator can provide arbitrarily broad coverage of both 
geography and demographics. The subsequent disaggregation is efficient because it is 
automatic. 

This makes it more cost-effective for publishers to deal with advertising aggregators than 
to directly capture advertising. Even though the advertising aggregator is taking a propor- 
tion of advertising revenue, publishers may find the change profit-neutral because of the 
greater efficiency of aggregation. 

Because of the finer targeting supported by the Netpage Network, publishers and advertis- 
ing aggregators have a larger advertising space to sell, leading to greater profits. The link- 
ing between advertising, detailed product information, \nd purchasing, and the 
corresponding measurability of consumer behavior, leads to greater profits from 
click-through fees and c-commerce commissions, benefiting publishers and advertising 
aggregators alike. 

Added revenue from these fees and commissions may even allow users' costs - printer 
ink, paper, and Internet access - to be fully subsidized. 



10.3.3 Netpage Points 



One possible subsidization mechanism involves maintaining an account of Netpage 
"points" for each user. 

Netpage points are earned in proportion to the amount of advertising the user accepts. 
Points are automatically deducted to cover the running costs of the printer. I he user can 
also redeem points for particular products and services offered on the Netpase Network, 
some of which may only be available in exchange for points. Points cannot be redeemed 
for cash. A user who has insufficient Netpage points can be billed directly for Netpa»e 
running costs. ' ^ 

Netpage points may also be used more widely to reward a user for making purchases via 
the Netpage Network. 



10.3.4 Netpage Newspaper as Portal 



The front page of a personalized newspaper delivered v ia the Netpage Network can con- 
tain a standard set of category buttons, including for example: 



Books 

Clothing 

Plectronics 

Pntertainment 

Pi nance 

Groceries 

Health 
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House and Garden 
Looking Good 
Music 
Restaurants 
Special Offers 
Sports 
Travel, and 
Emergency 



,he r h , k ,° S l T r SCrV,CCS rdate l ° thC CatC ^ 0r >- of a particular button can buy 
the right to be l.nked to that button. Several levels of linking are possible. An advertiser 
can buy the ent.re link, so that the button leads directly to the advertiser's page Or the 
advertiser can join a directory of advertisers who are linked to the button so that the 
advertiser s page is linked to a (usually small) directory which is linked to the button The 
publisher chooses how to deploy each button category and each button instance Th,„„ k 
_ is..er can segment the button space of their subscriber base in arbitrary wavs, for example 
by demographics, location, or simply proportionally to satisfy advertiser's who wish to 
purchase a subset of the space. 

Certain category buttons, such as "Emergency-, can link to non-advertising information 
specific to the user s location, and are provided as a service by the publisher. 

Web portals typically offer similar category buttons to advertisers. 
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1 Netpage Applications 

I he Netpage System can be used to deliver many of the same interactive computer appli- 
cations normally delivered via networked computer terminals such as Web terminals. 

In the Web paradigm, a series of Web "pages" displayed on the terminal screen provide the 
user interface to the application. In the Netpage paradigm, a series of printed Netpages 
provide the user interface to the application. 

'I he follow ing sections define a set of core Netpage applications, including user and pen 
registration, news subscription, electronic mail (e-mail), and various examples of elec- 
tronic commerce (e-commerce). In each case the application is described from the point of 
view of the user, in terms of the Netpage user interface seen by the user and the object 
model implicitly manipulated by the user through the user interface. 



1.1 Object Model Description 

Laeh object model is described using a Unified Modeling Language (UML) class diagram 
[3,9]. A class diagram consists of a set of object classes connected by relationships. Two 
kinds of relationships are of interest here: associations and generalizations. An association 
represents some kind of relationship betw een objects, i.e. betw een instances of classes. A 
generalization relates actual classes, and can be understood in the following way: if a class 
is thought of as the set of all objects of that class, and class A is a generalization of class B, 
then B is simply a subset of A. The UML does not directly support second -order model- 
ling - i.e. classes of classes. 



1 









posi' ion 




teacher 






tirricf Tabic 1 



Figure 1. Illustrative UML class diagram 

Lach class is drawn as a rectangle labelled with the name of the class It contains a list of 
the attributes of the class, separated from the name by a horizontal line, and a list of the 
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operations of the class, separated from the attribute list by a horizontal line. In the class 
diagrams which follow, however, operations are never modelled. 

An association is drawn as a line joining two classes, optionally labelled at either end with 
the multiplicity of the association. The default multiplicity is one. An asterisk (*) indicates 
a multiplicity of "many", i.e. zero or more. Each association is optionally labelled w ith its 
name, and is also optionally labelled at either end with the role of the corresponding class. 
An open diamond indicates an aggregation association Cis-part-of 1 ), and is drawn at the 
aggregator end of the association line. 

A generalization relationship ("is-a") is drawn as a solid line joining two classes, w ith an 
arrow (in the form of an open triangle) at the generalization end. 

When a class diagram is broken up into multiple diagrams, any class which is duplicated is 
shown with a dashed outline in all but the main diagram which defines it. It is shown with 
attributes only where it is defined. 

Figure 1 illustrates the subset of the UML class diagram notation used here 



1.2 User Interface Description 

Each application user interface How is illustrated as a collection of documents linked bv 
command arrows. A command arrow indicates that the target document is printed as a 
result of the user pressing the corresponding command button on the source page. Some 
command arrows are labelled with multiple commands separated by slashes (V's), indicat- 
ing that any one of the specified commands causes the target document to be printed 
Although multiple commands may label the same command arrow, they typically have 
different side-effects. 



In application terms, it is important to distinguish between Net page documents and 
Netpage forms. Documents contain printed information, as well as command buttons 
which can be pressed by the user to request further information or some other action. 
Forms, in addition to behaving like normal documents, also contain input fields which can 

be tilled in b> the user. The> provide the system with a data input mechanism. 

It is also useful to distinguish between documents which contain generic information and 
documents which contain information specific to a particular interaction between the user 
and an application. 

Fach document t\pe has a distinct iconic representation as shown in I iLUire ? 




Figure 2 Document icons used in user interface flows 
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may, of course, also be generated on -t he- fly by a Netpage Primer in response to user 
requests. 

User-spec i tie documents and forms are normally generated on the fly by a Netpage Printer 
in response to user requests. 



fixed information 

i variable information 



input field 



(^command button ^ 



\e>t of draqqab\e objects or attributes 



Figure 3. Page layout elements 

The Net pages which participate in a user interface flow are further described as abstract 
page layouts. A page layout may contain fixed information, variable information, input 
fields, command buttons, and lists of draggable objects and attributes, bach type of layout 
element has a unique style to differentiate it from the others, as illustrated in Figure 3. 

When a user interface flow is broken up into multiple diagrams, any document which is 
duplicated is shown with dashed outlines in all but the main diagram which defines it. 



1.3 Netpage User Interface Design Guidelines 

Like a Web application, a Netpage application is forms -oriented and transact ion -driven. 
I he application is largely decoupled from the user interface in the sense that it is only 
notified when the user submits a form, not while the user is filling in the form. When the 
time it takes to deliver a form is short, a transaction-driven system can approximate the 
behavior of a truly interactive svstem. Such is often the case with the Web. With the 
Netpage System, however, the time it takes to deliver a form is longer, but more impor- 
tant^, a form is not delivered in situ, but in the shape of another ph\sical piece of paper. 
The new page must be physically collected from the printer, which may not be proximate 
to the user, and even if the actual printing cost of the page is small, the user may perceive 
the user interface as being wasteful. 

It tli ere to re behooves a good Netpage application user interface to be more sparine in the 
printing of intermediate pages than an equivalent screen-based application, and to ensure 
that the content of each printed page is valuable to the user whenever a page is printed 

1.3.1 Input Feedback 

As with any computer application, it is important to provide the user with immediate feed- 
back on all input, so that the evolving state of the transaction is visible and not hidden. 

When a user fills in a Netpage form, immediate feedback is provided because the user uses 
the marking nib of the Netpage Pen. This applies to input in textboxes, checkboxes, signa- 
ture fields and the like When the user presses a com in and button, t he re is no immediate 
feedback on the page since the user uses the non-marking nib of the pen, but feedback is 
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nonetheless provided in the shape of the next form in the user interface series, or a transac 
uon receipt, or some other requested document. 

The user also uses the marking nib of the pen to input commands which don', require 
immediate interpretation, such as text editing markups. 

The following are examples of input typically made with the marking nib: 

• handwritten text 

• drawing 

• checkbox mark 

• signature 

• text editing command (strikeout etc ) 

• drag-and-drop (from list of objects, etc.) 

• drag-and-lasso (from palette of colors, line styles, etc.) 

I he following are examples of input typically made with the non-marking nib: 

• selection by circumscription 

• button press 

A Minimum Unit of User Interface Output 

A Netpage Printer prints a, a constant page rate, independent of whether pa S cs are printed 
on both sides or not, and independent of the pages' actual content. ~ 

A single page printed on both sides at high resolution in full color is therefore a useful 
I:™"" UmI of user ""^ ««'P««- This minimum unit of output has an mforma- 
'on-bcanng capacity equivalent to ten SVGA computer screens. So although a Netpa-e 
appicat.on ,s designed to produce less frequent output than the equivalent Screen-based 
X^Zp^ timCS " diSP ' ay " 3rea ° f S —^sed application when it does 

furthermore, previous page, printed as part of the user interface How continue to act as 

comn , mtCr,aCC ' a ' ,d " 1U ' llple a PP' icatio "s can be active simultaneously without 

competing for screen space and forcing the user to switch between them on the screen 
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Directory Navigation 

A directory is an organized list of named objects, [t may be presented in many ways: 
sorted alphabetically, arranged into named groups or topics, etc. The default presentation 
is specific to each kind of directory . A directory may also be searched. 

F or the purposes of navigation, an n-ary index tree is built above the directory , with each 
node containing enough index entries to fill two printed pages (i.e. a double-sided sheet). 
If two pages hold k entries, and the entire directory contains m entries, then the number of 
levels in the index is log k m. Tor example, if two pages hold 200 entries, and the entire 
directory contains 1,000,000,000 entries, then the number of levels in the index is 4. 

Eiach index entry specifics the starting and ending names on the two pages it is linked to, 
separated by "to", e.g. "Aardvark to AxolotT, "Ayatollah to Bernoulli 1 , etc. Each index 
entry acts as a hyperlink to the corresponding lower-level index node. The layout of the 
generic directory index page is shown in F : igure 7. 

If the entire directory fits on two pages, then it is presented directly w ithout an intervening 
index tree. 

I lie indexed directory class diagram is shown in f mure 20. 



index 


nodr 















Figure 4. Indexed directory class diagram 

I he presentation of the index entries of a terminal (bottom-level) index node, i.e. a node 
which refers to actual named objects, is specific to each kind of directory. 

I.ach page contains a - top • button which prints the root node of the index, an ■ up * button 
which prints the page's parent node in the index, and - first \ * previous \ - next - and 
last - buttons which print the first, previous, next and last nodes, respectively, of the 
page's index level. Particular navigation buttons don't appear if they have no mean in « for 
the current page, eg the page tor the first index node in an index level doesn't have 
' first - and * previous "* buttons 
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bach page also contains a <search " button which generates a search form which allows 
the directory to be searched. The contents of the search form are specific to each kind of 
directory. 

I he directory navigation user interface flow is illustrated in Figure 5. 
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Figure 5. Directory navigation user interface flow 



I he meaning of the standard navigation buttons on the directory index pane is described in 
Figure o. 
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Standard directory index navigation button 
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Figure 7. Generic directory index page 
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3 User Registration 



Much of the Netpagc System is predicated on the system's secure knowledge of user iden- 
tity. This allows the system to: 

• deliver personalized services to indiv idual users independent of location 

• support authenticated transactions 

• support secure payments 

• guarantee a high level of privacy 



3.1 User Object Model 



The user object model revolves around a uniquely identified user, as shown in the class 
diagram in Figure 8. 
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A user is authorized to use any number of printers, and printer may have any number of 
users. One of the user's printers is designated as the default printer, and subscriptions are 
automatically delivered to the default printer. Pages requested ad hoc, on the other hand, 
ate automatically delivered to the printer through which the user is interacting. Thus a 
publisher never records the id of a printer, but instead resolves the id when it is required. 

When a user subscribes to a publication, the publisher is authorized to print to the user's 
default printer. This authorization can be revoked at any time by the user. 

A Web terminal may be authorized to print to a Netpage Printer via the Netpage Network 
I his allows it to conveniently print Web and Netpage documents encountered while navi- 
gating the Web. 

A user may have any number of pens, but each pen is linked to a particular user. Each of a 
user's pens may be used interchangeably. 

Lach user also has a Netpage account which is used to accumulate micro-transactions; 
contact details, including name, address, and telephone numbers; global preferences, 
including privacy, delivery, and localization settings; any number of biometric records! 
containing the user's encoded signature, fingerprint etc.; and any number of payment card 
accounts, with which c-commerce payments can be made. 

'I he user's mailbox and contact list are described in Section 4. 



3.2 User Registration User Interface 



3.2.1 Help Page 

I he help page contain-, the following buttons uhich generate registration tonus: 

• add user 

• add pen to user 

• add pen to local user 

• authorize user on printer 

• authorize global user on printer 

• authorize terminal on printer 

3.2.2 User Registration 

A user must be registered with the Netpage Network, own at least one registered Netpage 
Pen. and be authorized on at least one Netpage Printer, before being able to mean imzfulk 
interact w ith the Netpage Network. 

I he - add user 1 button on the help page generates a user registration form (Fmure !0) 
which captures the user's contact details and privacy preferences. 

The pen used to submit the user registration form, if previously unregistered, is automati- 
cal k linked to the new user The user is automatically authorized to use the printer through 
which the registration takes place. 

A subset of the users authorized to use a printer are designated as administrators, and onlv 
the\ ma\ authorize the registration of additional users and pens on the printer 



20 October 



10 



Silverbrook Research 



Netpage Applications 



draft vO 2 



By default, every new user is an administrator on a particular printer until a printer-spe- 
citic privileges setting is enabled (via the help page). Thus a new user can be registered 
without having to be authorized by an administrator's pen until the privileges setting is 
enabled. This setting may never need to be enabled in a domestic environment, but may 
commonly be enabled in a corporate environment. User-specific privileges settings only 
appear on the registration form if the printer-specific privileges setting is enabled. 

When privileges are enabled, the user registration form must be authorized by an adminis- 
trator's pen. This is done by initialling the registration form in the appropriate place. 

I he user registration user interface flow is illustrated in Figure 9. 




Figure 9. User registration user interface flow 
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Figure 10. User registration page 
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3.2.3 Pen Registration 

Additional Netpage Pens may be linked to existing users. 



The <add pen to user> button on the help page generates an pen registration form (Figure 
12). The form contains name and nickname fields and a <submit> button. If the form is 
requested using the new pen, i.e. the pen to be registered, then the form is blank and must 
be filled in with sufficient name details to identity the user. If the form is requested using 
an existing pen, then the form is automatically filled in with the details of the existing 
pen's owner. 
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Figure 11. Pen registration user interface flow 

When the form is submitted, the name information is matched against the list of local 
users, i.e. users authorized to use the printer. Partial name information can be entered, such 
as just a first name or a nickname ! f the name information is ambiguous, t ho s> stem gen- 
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The <add pen to local user> button on the help page generates a list of all users authorized 
to use the printer (Figure 14), with an <add pen"* button next to each. The local user list is 
only available to administrators. 

When privileges are enabled, the pen registration form must be authorized by an adminis- 
trator's pen. This is done by initialling the registration form in the appropriate place. 

The pen registration user interface flow is illustrated in Figure 1 1 . 
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Figure 12 Pen registration page 
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Figure 13. Pen registration - matching users page 
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Figure 14. Pen registration - local users page 
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3.2.4 Authorize User on Printer 



A user may be authorized to use additional Netpage Printers. Once authorized to use a par- 
ticular printer, the user may interact with the printer using any pen linked to the user. 



The <authorize user on printer > button on the help page generates a user authorization 
form (Figure 16). The registration form contains name and nickname fields and a <sub- 
mit> button. If the form is requested using a pen owned by a local user, then the form is 
blank and must be tilled in with sufficient name details to identity the new user. If the form 
is requested using the new user's pen, then the form is automatically filled in with the 
details of the new user. 

submit 
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matching 
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Figure 15. User authorization user interface flow 



When the form is submitted, the name information is matched against the global user list. 
Partial name information can be entered, such as just a first name or a nickname If the 
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T he <authorize global user * button on the help page generates a list of all users on the net- 
work (Figure 18), with an <authorizc user - button next to each. The global user list is 
available to ail users, but only contains users who have elected to appear. 

When privileges are enabled, the user authorization form must be authorized by an admin- 
istrator's pen. This is done by initialling the authorization form in the appropriate place. 

The user authorization user interface flow is illustrated in Figure 15. 
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Figure 16. User authorization page 
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Figure 18. User authorization - global users page 
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3.2.5 Authorize Terminal on Printer 

The <authorize terminal on printer> button on the help page produces a terminal authori- 
zation token. The token is printed with information which allows the user of the Web ter- 
minal to register the Web terminal with the Netpage Network, allowing it to print to the 
Netpage Printer identified by the token. 

An administrator may revoke a terminal's authorization at any time. The <authorize termi- 
nal on printer> button is also only available to administrators. 




Figure 19. Terminal authorization user interface flow 
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4 Electronic Mail 

Netpage electronic mail (e-mail) provides a messaging service between Netpage users, it 
also supports message exchange with Internet e-mail users, and by extension, users of 
other e-mail systems interconnected with the Internet, such as corporate e-mail systems. 

Each Netpage user has a unique id within the Netpage System, and is also known to the 
system by their name, which may or may not be unique. Netpage e-mail is usually 
addressed by selecting a name from a list, for example from the global list of Netpage 
users or from a particular user's list of contacts. The user's nickname or alias helps disam- 
biguate similar names. A Netpage user usually doesn't have to know or specify another 
Netpage user's unique id. 

When a Netpage user wants to send e-mail to an Internet user, they must specify an Inter- 
net e-mail address. Similarly, when an Internet user wants to send e-mail to a Netpage 
user, they must specify the Netpage e-mail address. A Netpage e-mail address takes the 
form <userid>@ netpage . net. 



E-mail is a core service of the Netpage System. 



4.1 E-mail Object Model 

4.1 .1 E-mail User 

The e-mail object model revolves around an e-mail user. An e-mail user is either a 
Netpage user or an Internet user. 

The e-mail user class diagram is show n in Figure 20. 
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Figure 20. E-mail user class diagram 



4.1.2 E-mail 

I he e-mail itsel f consists of a number of pages. Fach page corresponds to a Netpage. The 
page structure is logical because the e-mail is delivered in the same form that it is com- 
posed, and it is composed by the sender page by page. 

Fach page contains digital ink. together with an\ number of attachments. An attachment is^ 
printed at its point of insertion, and may overflow onto the following page(s). 
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The sender of the e-mail is an e-mail user, as are any ruiniber of recipients and copy recip- 
ients (CCs). Blind copy recipients (BCCs) are not modelled here for clarity, but can obvi- 
ously be trivially accommodated. 

The e-mail has a explicit subject which is always convened from digital ink to text to sup- 
port interoperation with e-mail systems which don't directly support digital ink, such as 
e-mail systems on the Internet, and to streamline the presentation of mailbox contents 
within the Netpage System itself. 

The e-mail has a high priority flag which allows the both sender and the recipients to con- 
trol the timeliness of its delivery. 

The e-mail class diagram is show n in Figure 21. 
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Figure 21. E-mail class diagram 

4.1.3 Mailbox 

Incoming e-mails may be accumulated in the user's mailbox, printed on the user s default 
printer, incorporated into the user's daily newspaper, or any combination of these. The 
user can select, for example, to print high-priority e-mails immediately on receipt but to 
hold low-priority e-mails. 
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I he user may create any number of named folders within the mailbox, and either manually 
copy and move received e-mails between folders, or associate a folder with one or more 
e-mail contacts (including e-mail groups) to have e-mail received from those contacts 
automatically placed in the folder. E-mail received from contacts not associated with a 
folder is placed in a predefined "inbox" folder. All e-mail sent by the user is placed in a 
predefined "sent e-mail" folder for future reference. 

Each folder has three delivery options: "print all e-mail", "print high-priority e-mail", and 
"delete e-mail once printed". If a print option is selected, e-mail messages with the corre- 
sponding priority are printed on the user's default printer immediately on receipt. If the 
delete option is selected, e-mail messages are deleted from the e-mail folder once printed. 
Otherwise e-mail messages are held indefinitely in the e-mail folder, i.e. until manually 
deleted by the user. 

The mailbox class diagram is shown in f : igure 22. 



Each user has a list of contacts. The contact list provides a more convenient basis for 
selecting e-mail recipients than the global list of users, particularly since most user's con- 
tact lists will fit on a single double-sided printed page. The contact list also provides a 
basis for ignoring unsolicited e-mail. A privacy option allows a user to ignore e-mail not 
sent by a member of the contact list. 

A user may create any number of contact groups w ithin the contact list, and treat a group 
as a single contact for the purposes of addressing outgoing e-mail and control line the 
delivery of incoming e-mail. 

Groups may themselves contain groups, and both individual contacts and eroups mav be 
members of multiple groups as well as the top-level contact list itself. 

The contact list cla^s diagram is shown in f igure 23. 
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Figure 23. Contact list class diagram 
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4.1.5 



Barred User List 



Rather than accepting e-mail only from known contacts, a user may choose to bar individ- 
ual users. The barred user list records individuals from which the user refuses to accept 
e-mail. 

I he barred user list class diagram is show n in Figure 24. 
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Figure 24. Barred user list class diagram 

4.2 E-mail User Interface 

4.2.1 Send E-mail 

[■.-mail is sent using the outgoing e-mail form (see Figures 27 and 28). The e-mail form is 
alwaxs printed uith the sender's name already specified, since the sender's identity is 
known from the pen used to request the e-mail form. If the e-mail form is requested by 
pressing a user-specific <e-mail> button on some page, then the e-mail form is printed 
with the recipient's name already specified. On the other hand, if the e-mail form is 
requested from the help page, then the recipient is unknown and is left blank on the form. 
I he user must specify at least one recipient before the e-mail can be sent. 

The user can also pre-address an e-mail form by pressing the <e-mail to contacts> button 
on the help page. This elicits an add recipients form (Figure 29), which lists the user's 
contacts. Fach contact has a To - and a • ("(/ • checkbox as well as an < e-mail - button. 
I he e-mail button elicits an e-mail form addressed to that user "the - e-mail selected • but- 
ton at the bottom of the page elicits an e-mail addressed to all the user's w Iukc io ■ or 
( (' - checkboxes ha\e been checked 

I he e-mail form initially consists of a double-sided page. I he front (Figure 27) contains 
fields for the names of recipients and copy recipients, for the subject, and for the both of 
the e-mail. The back (f igure 28) contains a field for continuing the bodv of the e-mail. 
An> recipient names (or addresses) written by hand are converted from digital ink to text 
for lookup purposes The subject is also converted, for presentation purposes, as discussed 
earlier I he bod> is retained as digital ink, to allow handwritten text and diagrams etc to 
be deh\en*d in .1 'I'lifu'" 1 ■ ■ 
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but the additional page is logically linked to the original e-mail. The <discard page> but- 
ton at the bottom of every page discards the corresponding page from the e-mai! (but not 
both sides of the double-sided page). The entire e-mail is reprinted with the discarded page 
removed and subsequent pages, if any, moved to close the gap. A blank page added to 
make the page count even, if necessary. 

liach printed e-mai! form corresponds to a separate e-mail instance, the uniqueness of 
which is indicated by the sender's name, date and time printed at the top of every page. 
Once an e-mail has been sent, it canT be sent again. A copy, however, can be easily made, 
edited, and sent. The standard <print> button on every page of the e-mail form prints 
another copy of the e-mail form, corresponding to a new e-mail instance. Both the original 
and the copy can be edited further and sent independently. 



e- m^il 
contact 




Figure 25. E-mail user interface flow 



Ie\t in the ■ recipients , - cop> recipients' - and ■ subject * can be struck out with the pen 
to remove recipients, subject text, etc. The <print> button produces a copy of the e-mail 

form with the struck-out text removed. 

I he <attach> button at the bottom of every page attaches the current selection at the cur- 
rent end of the e-mail body. The entire e-mail is reprinted with the attachment included. 
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Additional pages are automatically added to the e-mail to accommodate the attachment. 
I he attachment can consist of anything which can be selected on any Netpage. 

I he <add recipients* button adjacent to the Recipients - and * copy rccipicnts> fields at 
the top of the first page of the e-mail form elicits an add recipients form (Figure 29), with 
the <subject> Held reflecting the subject of the e-mail form. The -e-mail > and <e-mail 
selected- buttons on the add recipients form elicits a copy of the e-mail form with the 
additional recipients and copy recipients added. 

The <send> button at the bottom of every page sends the entire e-mail. If the name of any 
recipient is unknown to the system, then the e-mail form is reprinted with the offending 
name printed in red, together with an error message indicating the problem. 

If the name of any recipient is ambiguous, then a list of matching users is printed, each 
w ith a checkbox allowing it to be selected. The <e-mail> button at the bottom of the form 
reprints the e-mail form with the recipient names suitably updated. 



(«a*nt'i<juous 




Figure 26. E-mail recipient exception user interface flow 

The overall e-mail user interface flow is illustrated in Figure 25. The 
exception user interface flow is illustrated in Figure 26. 



e-mail recipient 



Silverbrook Research 



Netpage Applications 



r — — — — — — — — — — — _________ — — — — — — — —— _______ r ____ 

I E-mail from eender at time & date' ' " : \r a rf. 



Jo 


rec\p\ent(e) 






jj Friohty 


CC 


copy reclpien t(e) 






| ^add recipients^ 


Subject 


subject L 


• 


iine color palette* 




• 


line width palette 



e-mail body 



Figure 27. Outgoing e-mail first page 
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Figure 28. Outgoing e-mail second or subsequent page 
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Figure 29. Add recipients page 
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4.2.2 Receive E-mail 

l:-mail is received in the form of an incoming e-mail document (Figures 30 and 31), 
which has the same page structure and content as the corresponding outgoing e-mail form. 

I he <repl\ > button at the bottom of every page of the e-mail document produces an out- 
going e-mail form (Figures 27 and 28) addressed to the sender of the incoming e-mail, and 
with the subject reflecting the subject of the incoming e-mail, but with k Mn reply to:" added 
as a prefix. 

The <reply to all > button produces an outgoing e-mail form addressed to the sender of the 
incoming e-mail, as well as to all of its recipients and copy recipients. 

The <forward> button produces an outgoing e-mail form with no recipient, and with the 
subject reflecting the subject of the incoming e-mail, but w ith "Forwarded:" added as a 
prefix. The body of the incoming e-mail is also copied to the body of the outgoing e-mail. 

The <forward to contacts> creates, but doesn't print, an outgoing e-mail in the same way 
as the < forward > button, and then implicitly invokes the <add recipients > command on it. 
I his elicits an add recipients form in the usual way (Figure 29), with the <subject> field 
reflecting the subject of the forwarded e-mail. 

The <add to contacts > button adds the sender to the recipient's list of contacts and pro- 
duces an edit contacts form reflecting the updated contact list. Contact list editing is 
described in more detail in Section 4.2.3 on page 34. 

I he <bar> button adds the sender to the recipient's list of barred users and produces an 
edit barred users form reflecting the updated barred user list. Barred user list editing is 
described in more detail in Section 4.2.5 on page 43. 
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Figure 30. Incoming e-mail first page 
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Figure 31. Incoming e-mail second or subsequent page 
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4.2.3 Edit Contact List 

I he contact list is edited using the edit contacts form (Figure 33). Because the structure of 
an individual contact group is the same as the structure of the top-levei contact list, the 
same form is used to edit a contact group. The form is obtained by pressing the <edit con- 
tact list > button on the help page. It is also printed whenever a contact is added to the con- 
tact list, for example when the <add to contacts> button is pressed on an incoming e-mail 
to add the sender to the contact list. 

Each entry on the contact list form shows the name on the contact, and the current e-mail 
folder in which e-mail from the contact is placed. Each entry has a checkbox which allows 
the contact to be selected, an <info> button which elicits a page of information about the 
user (figure 44), and a <set mailbox> button which allows the e-mail folder associated 
with the contact to be changed via the set e-mail folder form (Figure 35). 

The <delete selected> button at the bottom of the form deletes the selected contacts from 
the contact list (or contact group). The form is reprinted with the deleted contacts 
removed. 

The < copy to group> and - move to group> buttons at the bottom of the form allows the 
selected contacts to be copied and moved to a particular group via the copv contacts form 
(f igure 34). 
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ie contact list editing user interface flow is illustrated in figure 32. 
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Figure 32. Edit contact [list.group] user interface flow 
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Figure 33. Edit contacts page 
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1 he copy (move) contacts form provides a * new group name - field with an associated 
- copy to new > (<move to new>) button. This allows a new group to be simultaneously 
created and selected as the destination for the copy or move. 
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Figure 34. Copy contacts first page 

The set e-mail folder form lists all of the folders in the user's mailbox, each with a <set-> 
button which sets the folder as the contact's e-mail folder When the ■ set - button is 
pressed, the edit contacts form is re-printed to reflect the new folder associated with the 
contact 
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I he set e-mail folder form provides a <new e-mail folder name ■ Held with an associated 
<set new > button. This allows a new folder to be simultaneously created and selected as 
the contact's e-mail folder. The new folder has an associated set of checkboxes which 
allow the folder's e-mail delivers options to be specified. 
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4.2.4 List Mailbox 

The contents of the user's mailbox are listed using the mailbox form (Figure 37) and the 
e-mail folder form (Figure 38). The mailbox form lists all of the user's e-mail folders, 
each with a set of checkboxes which reflect the folder's e-mail delivery options and allow 
the delivery options to be changed. Each folder entry also has a checkbox which allows it 
to be selected. The <count> next to each folder name indicates the number of un-printed 
(i.e. un-read) e-mails in the folder. 

The <delete selected-* button at the bottom of the form deletes all selected folders from the 
mailbox. The mailbox form is re-printed with the deleted folders removed. 

The <update options > button at the bottom of the form applies any changes made via the 
checkboxes to the folders 1 delivery options. 

Each folder entry also has <list> button which lists the contents of the folder via the e-mail 
folder form (Figure 38). 

The mailbox listing user interface flow is illustrated in Figure 36. 
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[copy, move] 
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Figure 36. List mailbox user interface flow 

I he e-mail folder form lists all of the e-mails in the folder. Each sirmle-line e-mail entrv 
shows the sender of the e-mail, the subject of the e-mail, and the date and time the e-mail 
was received. The list can be sorted by sender, subject, or date and time. Each column 
which is not the current sort column has a <sorf> button which allows the list to be sorted 
b\ that column. When a ■ sort - button is pressed the e-mail folder is soiled b\ the selected 
column and the form is re -printed 
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Figure 37. Mailbox page 

U here an e-mail is a repl> to another e-mail, the subject of the repK \\w\ e-mail h indented 
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hach c-mail entry has a checkbox which allows it to be selected, as well as a print button 
which allows the entire e-mail to be printed in the form of an Incoming b-mail (bmures 30 
and 3 1). 

I he <delete selected > button at the bottom of the form deletes the currently selected 
e-mails. The e-mail folder form is reprinted with the deleted e-mails removed. 
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Figure 38. E-mail folder page 



i-0 (October 1 v,', 



Silverbrook Research 



Netpage Applications 



draft vO 2 



The <copy selected • and <move selected > buttons at the bottom of the e-mail folder form 
allow the currently selected e-mails to be copied or moved to a different e-mail folder via 
the copy e-mail form (Figure 39). 
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Figure 39. Copy e-mail first page 
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Ihc copy (move) e-mail form provides a <new e-mail folder name> Held with an associ- 
ated <copy to new> (<move to new>) button. This allows a new folder to be simulta- 
neously created and selected as the destination for the copy or move. ITie new folder has 
an associated set of checkboxes which allow the folder's e-mail delivery options to be 
specified. 
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4.2.5 Edit Barred User List 

I he barred user list is edited via the edit barred users form (Figure 41). The form is 
obtained by pressing the <edit barred user list> button on the help page. It is also printed 
whenever a user is barred, for example when the <bar> button is pressed on an incoming 
e-mail to add the sender to the barred user list. 

The form lists all of the users in the user's barred user list. Each has a checkbox which 
allows it to be selected. 

The <unbar selected > button at the bottom of the form deletes the selected users from the 
barred user list. The form is re-printed with the deleted users removed. When a user is 
deleted from the list that user is no longer barred. 

'1 he barred user editing user interface How is illustrated in Figure 40. 




Figure 40. Edit barred user list user interface flow 
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Figure 41. Edit barred users page 
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4.2.6 Global User Directory 

The global user directory, since it is large, is navigated via a directory index (as described 
in Section 2 on page 6). It is obtained by pressing the <global user directory^ button on 
the help page. 

The global user directory page (Figure 43) contains an alphabetical list, by family name, 
of users in the name range covered by the page. Eiach entry has three associated buttons. 
The <info> button produces a user information page (Figure 44), the <e-mail> button 
generates an outgoing e-mail form (Figures 27 and 28) addressed to the corresponding 
user, and the <add to contacts> button adds the user to the user's contact list and prints the 
updated contact list (Figure 33). 

The global user directory page also contains the standard index navigation buttons. 

The global user directory user interface flow is illustrated in Figure 42. The index user 
interface flow and associated page layouts are as described in Section 2 on page 6. 

help 
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Figure 42. Global user directory user interface flow 

I he ibci information page ( f' igure 44 J contains the user's contact details as pro\ ided dur- 
ing user registration, subject to the user's privac> preferences. It also contains - e-mail ■ 
and - add to contacts"* buttons which work in the usual way. 

I he • Internet e-mail address ■ is only included if the user information paue is printed via 
the contact list for an Internet contact (as opposed to a Netpage contact). 
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Figure 43. Global user directory page 
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Figure 44. User information page 
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4.2.7 Business Card 

A Netpage business card (Figure 46) serves a dual purpose. While it conveys the user's 
contact details in the usual way, it also acts as an e-mail authorization token. When the 
recipient of the card presses the <add to contacts> button on the card, the owner of the 
card is added to the recipient's contact list in the usual way, but the recipient is also added 
to the owner's contact list, authorizing the recipient to send e-mail to the card owner, 
should the card owner only accept e-mail from known contacts. The token only allows the 
recipient to be added to the owner's contact list once. 

The card also contains an <e-mail> button which generates an outgoing e-mail form 
addressed to the card ow ner in the usual way. 

The business card user interface flow is illustrated in Figure 45. 
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Figure 45. Business card user interface flow 
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Figure 46. Business card 

Since normal Netpage Printers don't support business card paper stock. Netpage business 
cards are provided by a service bureau on request The business card request form (Fig- 
ure 48) allows the user to specif\ which contact details to include on the business card, an 
optional background texture, and the number of cards required. Once the request is sub- 
mitted, the cards are automatically printed by the bureau, delivered b\ mail to the user's 
address, and the small cost charged to the user's Netpage account. 
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Figure 47. Business card request user interface flow 
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Figure 48. Business card request page 
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4.2.8 Add Internet Contact 

An Internet e-mail user can be added to a user's contact list using the Internet contact 
registration form (Figure 50). Since an Internet user is not otherwise known to the 
Netpage System, the form allows full name details to be specified, in addition to the Inter- 
net user's e-mail address. 

The Internet contact registration user interface flow is illustrated in Figure 49. 




Figure 49. Internet contact registration user interface flow 
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5 Purchasing 

The Netpage System provides efficient mechanisms for merchants to offer goods and ser- 
vices to customers, and for customers to select and pay for those goods and services. 

A customer can order directly from a catalog browsed and printed dynamically via 
Netpage, as well as from a pre-printed catalog delivered by traditional means. 

The customer's dynamic signature, in combination with the customer's uniquely identified 
pen, authent.cates the purchasing transaction in a way which is impossible to forge If the 
customer chooses to use the Secure Electronic Transaction (SET) payment mechanism 
bu.lt mto the Netpage System, then the customer's payment card details are never revealed 
to the merchant, ensuring that card details can neither be accidentally nor maliciously 
compromised. 

The purchasing mechanisms described in this section are generic, and apply to the pur- 
casing uf myriad kinds of goods and services. 

5.1 Purchasing Object Model 

The purchasing object model revolves around merchants and customers. 

Each merchant has a globally unique identifier, as well as a name and other details. Each 
customer .s related to a particular merchant, and has an identifier which is unique within 
the scope of the particular merchant. Each customer represents a particular Netpage user 
to the merchant. A Netpage user, on the other hand, can be the customer of anv number of 

merchants. 

Each merchant offers a range of products for sale, ty pically presented in the form of an 
organized catalog. Any Netpage user can browse the merchant's catalog, but the user must 
become an explicit customer of the merchant before being able to complete an order with 
the merchant. 
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The merchant class diagram is shown in Figure 51, while the customer class diagram is 
shown in Figure 52. & 
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Figure 51. Merchant class diagram 

The name and description of a customer derives from the details of the corresponding 

Netpage user. K ,u,u s 



Each customer has a history of payment methods and shipping addresses with the mer- 
chant. The most recent of each is recorded and used as the default on any new order. 

Fach customer has an account with the merchant which records the net balance of all 
invoices and received payments. Account payments, i.e. payments not associated with a 
particular invoice, are associated directly with the customer account 



accounz 
payment 



Jate & t.rre 
p'ment mtnd 
amount 



Netpage 
merchant 



Netpage 
user 



CUS-torrer 

account 



default 



pay men 



default 



^ripprtd 
addre^jij 



name 

address 

telephone 



Figure 52. Customer class diagram 
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Each customer has a history of orders, each in a particular state of completion as reflected 
by its status. The order class diagram is shown in Figure 

Each order is uniquely identified within the scope of the merchant. 

Each order has a status which indicates whether the order is pending (i.e. not yet submit- 
ted), active, partially or fully shipped, cancelled or completed. An order is not completed 
until full payment is received. 

Each order consists of a number of order items, each of which specifies the quantity of a 
particular product offered by the merchant. Each order item also includes a backorder 
quantity and a quantity shipped. The backorder quantity is used to register a claim on 
future stock when there is insufficient current stock to fully satisfy the order item. The 
quantity shipped is used to record the cumulative quantity actually shipped. 

Each order is associated with a particular payment method, shipping address, and shipping 
method. 
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merchant uill he specific to that merchant, and rna> be different for different customers of 
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When the Netpage account payment method is used, the user's Netpage account is deb- 
ited, and the transaction appears on the user's Netpage account statement. 

When the customer account payment method is used, the user's customer account is deb- 
ited, and the user is invoiced for payment. 

When the pro forma invoice payment method is used, a pro forma invoice is issued 
request.ng pre-payment of the order. When payment is received, the goods are shipped. 

When the cash on delivery payment method is used, payment is expected when the ooods 
are del ivered. s 

When the SET payment card payment method is used, the corresponding SET payment 
card Imked to the user is used to make payment according to the normal SET protocol. 

When the paymem d paymem method js used> ^ spedfied pa ^ ent ^ ^ 

make payment accord.ng to the normal protocol between the merchant and their acquire 
bank or institution which handles the particular payment card type. 

Invoice and pro forma invoice payments are normally made with a check or money order 
^ash on delivery payments are normally made with a check or cash. 
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Figure 54. Payment method class diagram 

The set of shipping methods which appl.es to a particular order depends on whether the 
order s shipping address is domestic or international with respect to the location of the 
merchant s warehouse Typical domestic shipping methods include standard shipping, sec- 
ond day a.r, and next day air. Typical international shipping methods include standard 
shipping, airmail, and international priority. 
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The 



most common shipping methods are shown in the shipping method class diagram in 
Figure 55. 
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Figure 55. Shipping method class diagram 
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5.2 Purchasing User Interface 



The status of the order determines the allowable set of operations on the order. While the 
order is pend.ng, the user can browse the merchant's catalog and add items to the shopping 
cart which represents the order. After the order is submitted, the user can still amend the 
payment method, shipping address and shipping method until the order is fully shipped A 
change obv.ously only applies to items not yet shipped when the change is made. The user 
can delete order .terns or change the quantity ordered until the items are actually shipped 
Quantity increases are subject to availability and payment approval 




t3ub-nit 



edit 




Figure 56 Online purchasing user interface flow 
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The overall purchasing user interface flow is illustrated in Figure 56. 

5.2.1 Locate Merchant 

A merchant can be located by browsing or searching the global Netpage directory or the 
user's own bookmarks. A merchant may also be encountered in the form of an advertise- 
ment in a Netpage publication. Both a directory entry and an advertisement typically link 
to the merchant's main page. This in turn links to the merchant's product catalog. If the 
user already has a copy of the merchant's main page from a previous encounter, then the 
catalog is immediately available. 

5.2.2 Browse Catalog 

A small product catalog is typically organized into a set of categories. The main catalog 
page simply lists the categories, and each category page lists all of the products in the cat- 
egory. This is indicated in the user interface flow in Figure 56. 

If the product catalog is large, then it may still be organized into categories, but it may be 
impractical to print the entire contents of a category when a user w ishes to browse the cat- 
egory. Instead the directory navigation techniques described in Section 2 can be used. A 
catalog search will often be specific to the kinds of products represented in the catalog. 

5.2.3 Order Item 

Many applications involve the user maintaining a collection of items. In the screen-based 
version of the application, the user's collection is typically shown every time an item is 
added or removed. The user therefore receives immediate feedback when the collection 
changes. The collection may reflect an e-mail recipient list, a shopping cart, etc. 

In the Netpage version of the application, it is less practical to re-print the collection every 
time it changes. Item deletions and quantity changes can be marked up visibly, and so do 
not necessarily require immediate re-printing. Item insertions, however, are more prob- 
lematic. In some cases the entire source list can be used as the input form, allowing it to be 
marked up visibly. This may be particularly apt if the source list already represents a 
user-specific subset of the complete source list, e.g. last week's shopping list rather than 
the grocery store's complete range. There will be cases, however, where the user will pri- 
marily be making selections from a complete catalog. The catalog may even be many hun- 
dreds of pages long, offset printed, and delivered in the mail. Since it is intended for 
multiple use, it is not suitable for visible markup. 

One way to order via a large catalog is to print intermediate product-specific panes. Fach 
catalog entry has both an <mfo - and an <-'buy > button. The - info ■ button elicits a 
tull-page description of the product with a quantity field which, if filled in, adds the prod- 
uct to the shopping cart and' provides a visible record of the item order. This avoids hav ing 
to print the contents of the shopping cart every time an item is added to it. The <buv> but- 
ton elicits a similar full-page description of the product but automatically adds it to the 
shopping cart. It contains a quantity field, already filled in with the quantity ordered by 
default, which can be used to vary the quantity ordered. 
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The product page is visibly linked to the order itself because it is tagged with the name of 
the current user, the order transaction number, and the order date and tir 



i me. 



5.2.4 Complete Order 



The shopping cart represents the contents of the current order. The shoppin* cart is acces- 
sible from any merchant page, whether the page is pan of the current order form or not 
I he shopping can provides a convenient place to review the order and make final changes. 

The checkout is also accessible from any merchant page. It provides a place to specify 
shipping and payment information, and finally to confirm the order. 

The checkout form is broken up into five sections: shipping address, shipping method 
payment method, order items, and order confirmation. Sections have default selections 
based on the user's most recent behavior. 

The order confirmation section summarizes the order and captures the user's signature It 
shows the current shipping address, shipping method, payment method, and order total. 

Any change the user makes to the order on the checkout form is reflected on the form 
since the change is made with the inking pen. It is not reflected, however, in the order 
summary m the order confirmation section. To produce a consistent checkout form, the 
user can simply press the <update> button at the bottom of the form after making a num- 
ber of changes. An updated checkout form is immediately printed. 

The <submit> button, when pressed, submits the order. Any handwritten change to anv 
item information page, shopping can page, or checkout page which is pan of the order 
form is automatically interpreted and applied to the order. The order receipt, printed in 

response to the order submission, reflects the actual contents of the order. 

The checkout form consists of a header (Figure 57), a section for selecting a previously 
used shipping address or specifying a new address (Figure 58), a section for selecting a 

shipping method (Figure 59), a section for selecting a previously used payment method or 
specifying a new payment card (Figure 60), a section for reviewing the list of order items 
and making last-minute changes (F.gure 6 I ), a section for signing the order to confirm 
payment (Figure 62), and a footer which contains the <update> and <submit> buttons 
(Figure 63). 

The printed checkout form will vary in length depending on the number of previously 
used shipping addresses and payment cards, and the number of items in the order. In man'v 
cases it will tit conveniently on two pages printed on a single sheet of paper 

Ihe order receipt lists the items ordered and summarizes the order parameters in much the 
same way as the order confirmation section of the checkout form (Figure 62). 



Checkout 



I °_ rdG ^ order tf. customer, time & date \ \ # 0 f # ] 




Figure 57. Checkout page header 
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( 1) Shipping Addreee 
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Figure 58. Checkout page shipping address 
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Figure 60. Checkout page payment method 
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(5) Confirm Order 
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Figure 62. Checkout page order confirmation 
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and details of individual orders. They can edit any order which has not been fullv shipped 
as described earlier. 

The order list gives the status for each order. Pressing the <info> button associated with 
each order yields a copy of the order receipt, but with an updated status for each order 
item. The status of an order item derives from the status of the order it is part, as well as 
the presence of shipment items and backorder items associated with the order item 



5.2.6 Edit Order 



The order receipt allows the quantities of individual order items which are still active to be 
altered. Once alterations are made, the user must press the <submit> burton. An updated 
receipt is produced. H 

To edit order parameters, the user can press the <edit> button on the order receipt This 
produces an order editing form which is similar in structure to the checkout form Once 
alterations are made, the user must press the <submit> button. An updated receipt is pro- 



5.2.7 Receive Shipment 



The shipment is accompanied by payment receipt, or, if the payment method is "cash on 
delivery", an invoice which allows the user to pay the delivery person bv traditional means 
(e.g. cash, check, or payment card not processed through Netpage), or us in* a Netpa«e 
payment method listed on the form, simply by marking the appropriate payment method 
and signing the form with a Netpage pen. 

A customer can therefore defer authorizing payment until the goods are delivered even 
when using a payment method which allows the Netpage Svstem to obtain immediate 
online payment authorization. Conversely, the delivery person can verify pavment before 
handing over the i>oods. 
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6.1 



Folder Hierarchy 

Folder Hierarchy 



There are several instances where a Netpage user is able to organize a set of items into a 
hierarchy of folders. Examples include the user's contact list, favorites list, etc. 

In each case the folder hierarchy consists of a root folder containing any number of folder 
entries. Each folder entry is either itself a folder containing further folder entries, or is a 
link to an actual folder item. 

Folder entries may in general be members of more than one folder, although particular 
applications may not allow this. Both folders and folder items are typically named, and a 
folder is often presented to a user sorted by name, although sorting by other attributes can, 
of course, be equally valid. 

When a folder item has an efficiently-retrievable name, and the folder is required to reflect 
the most up-to-date version of the name, then the folder item link is typically not named, 
and the name is instead retrieved from the folder item itself when required. 

Each folder hierarchy is maintained by the Netpage Registration Server on the user's 
behalf, as part of the user information maintained by the server 

The folder hierarchy class diagram is shown in Figure 64. 
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Figure 64. Folder hierarchy class diagram 
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Navigation Aids 



7.1 Favorites List 



Each Netpage user is allowed to maintain a list of "favorites" - links to useful documents 
on the Netpage Network. The list is maintained by the Netpage Registration Server on the 
user's behalf, as part of the user information maintained by the server. The user can orga- 
nize the list into a hierarchy of folders, edited and navigated in much the same way as The 
contact list described in Section 4. 

The favorites list class diagram is shown in Fiuure 65. 
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Figure 65. Favorites list class diagram 

Each printed page of the favorites list contains three buttons for addins a favorite to the 
list: <add selected documents <add selected page> and <add selected lmk>. The user can 
select a region of a page in the usual way (i.e. by circumscription) and then press one of 
the three buttons The <add selected document button adds a link to the document which 
contains the selected page. The <add selected page> adds a link to the selected page. The 
<add selected hnk> button adds the first hyperlink contained in the current selection. This 
allows a document to be added to the favorites list without printing that document. 

In each case the link is added to the root favorites list or a specific favorites folder depend- 
ning on whether the page the button is pressed on contains a portion of the root list or a 
portion of a specific folder's list. The help page also contains the three buttons for adding 
a favorite. These buttons always operate on the root list. 

Each time a favorite is added a single double-sided page is printed showing the part of the 
list containing the new favorite. 

Each entry in the favorites list is labelled with the name of the document. If the link is to 

an individual page, then the page number of that page is also shown 
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If the user attempts to add a hyperlink which only makes sense in the context of its origi- 
nal location (e.g. a <submit> button on a form), then the operation is rejected with an 
appropriate error report. 



7.2 History List 

The Netpage Registration Server maintains a history list on each user's behalf, as part of 
the user information maintained by the server. The history list contains links to all docu- 
ments accessed by the user through the Netpage System. The user can navigate the list by 
date and time, search the list by document name, delete entries from the list, and disable 
the server from maintaining the list. 

The history list class diagram is shown in Figure 65. 
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Figure 66. History list class diagram 
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8 Drawing and Painting 



The marking n,b of a Netpage Pen is typically a ballpoint which produces a fixed-width 
me of a s.ngle fixed color. The Netpage System can nonetheless be used to draw and paint 

faceTrr y US '" 8 V3 - iOUS ' ine StylCS ' C °' 0rS ' 3nd bmSheS ' Like in a g ra P hica > «er inter- 
S^h u* ° r P / mtmg ap P lica,ion - the Net P a 8 e equivalent presents a palette of styles 
vvh.ch can be used to draw or paint. The colors and styles used only become visible when 
tne drawing or canvas page is reprinted. 

Operation can be modal and modeless. When operating modally, the user simply selects 
co o s and styles from the palette and draws or "paints" (by drawing) in the usua" way on 

^T: C C \ Th 7 age ' ThCre ^ "° indiCati ° n ° f CUtTent "-° de " -'or, "pen 
iZs 1 i, h § ° r Pa J nt ' ng COnS ' StS merely of fixed -width and fixed-color drawn 

nt mode u P3Se 15 K Pnnted - ,f ^ h3S 3n LC ° then the LCD can indica * the cur- 
rent mode. However, there is no h.story of mode changes on the page itself. 

nTr h AV P T tinS modeless 'y- color ' P en and b ™h attributes are applied to a pan of the pic- 
ture after that part ,s drawn. Attributes can be applied in a modeless way by using a lasso 



8.1 Lasso 



?nr!h7 y SUP , P ° n m ° deIeSS Pe " attHbUteS iS t0 j ° in elements of the drawing to nen 

am - £u IT ' n ,l ^ ? e USmg / laSS ° dr3Wn WitH thC PCn - The ,a " of the lasso * Joined tothe 
attr bate m the palette and the loop of the lasso circumscribes the part of the drawing to 

which the attribute is being applied. = 

toV™° ' S ( m0de ' eSS Since il P r ° vides a S ra P hical representation of applying an attribute 
to a pan of the drawing. Lassos can also be drawn with the pen in non-marking mode and 
r,prmt can be requested (or can occur automatically) after each attribute is applied 
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9 Viewing 3D Objects 

When an application prints a view of a three-dimensional object on a Netpage a number 
of controls can be provided to a.low the user to change the view of the object A stlf 
controls is pnnted next to the image of the 3D object, and the user marks the coma s Id 
presses the <reprint> button to obtain a copy of the page containing the updated view. 

m h ,n 3 h °r bje ?- iS Pr ° jeCted £ ° 2 ° accordin g t0 a set of viewing parameters. These are nor 
mally denned m terms of a virtual camera, and typically include the following: 

• viewpoint, i.e. the point in 3D at which the camera is looking 

• camera position, i.e. the point in 3D where the camera is located 

• camera m ist, i.e. the rotation of the camera about the viewing vector 

• view plane distance, i.e. the distance from the camera position a!on° 
the viewing vector to the plane onto which the 3D scene is projected 

• view piane window, i.e. the 2D window on the view plane to which 
the scene is clipped 

• projection type, i.e. parallel, perspective, etc. 

The camera position is sometimes specified using intuitive parameters such as pan and tilt 
where pan rotates the camera about the view point in the x-y plane, and tilt rotates the 
camera pos.fon about the view point in a plane which contains the camera position and 
tne view point and is perpendicular to the x-y plane. 

The 2D projected image of the 3D scene can also be subject to 2D viewing parameters 
such as translation and scale, within the view plane window. 

3D viewpoint controls can be implemented in many ways, including the following: 

• viewing direction: a sphere surrounds a small copy of the object, with pan and tilt rin-s 
marked on the surface of the sphere (see Figure 67); the user specifies a new viewin<> 
direction by marking a new pan and/or tilt on the rings; the current pan and tilt setting 
are indicated on the rings, and the current viewing direction is indicated by a point on 
the surface of the sphere; the object can either be shown from a standard viewing 
direction or the current viwing direction. & 

• viewing direction: a small copy of the object is presented from opposite sides with a 
transparent shaded hemisphere surrounding each copy; the user specifies a new view- 
ing direction by marking a point on either hemisphere 

• 2D translation and scale: the user circles an area of interest: the center of the circle 
specifies the translation, and the bounding box specifies the scale 

• camera position: a camera icon is shown in the position of the camera in the view of 
the object; the user specifies a new camera position by drawing a pen stroke from the 
camera icon to the desired position; the camera keeps pointing at the view point. 

• camera twist: a handle is attached to the camera icon; the user specifies a new camera 
twist by drawing a pen stroke from the handle to the desired position 
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current 




Lilt control 



Figure 67. 3D view control 



Viewing parameters can also be specified via virtual sliders and numeric input. However, 
these forms of input are less intutitive and less user friendly, and so are more suited to pro- 
fessional applications where accuracy may be required. 

Previously printed pages show a history of views. Unlike screen-based 3D navigation sys- 
tems, the user cannot get lost because previous views of the scene are always available. 
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Authoring 
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10 Authoring Transition 
Strategies 



10.1 Overview 



fv U a l!? 8 t ,he t ar ' y S ' ageS ° f NCtpage NetW ° rk de P lo y«"ent. publishers and merchants will be 
anl ™S C ° StS T d bCnefltS ° f partici P ati "g- In order ^ encourage such participation 
and grow the network, u ,s .mportant to lower as far as possible any barriers to entry 

lTs° t T-l he -uT Se Pr0, T ,S ' SCrVerS netW ° rk infrast ^ture will be in place from 
and com ,lme t0 dCVel0P generk high -' cvel su PP° n for Publishing, advertisino 

and commerce. Early entrants may therefore be faced with significant development costs 

WorldTidew'h 8316 th ' S P r ° blem is 10 levera g e development efforts occurring in the 

We with LxJ T ^ .^T NCtWOrk " 3 mediUm Which is hi § h 'y - m P at " 
l m V" g Pn med ' a - ,C ,S 3,SO a " interactive medium which is highly com- 

patible w,th the Web. It is therefore well placed to exploit recent efforts to re lose 
pnnt-onented editorial content on the Web, and is also well p| aced t0 exploit Web applica- 
tions defined at a sufficiently abstract level, as is now starting to become the norm 

wS re anH 8 rh T'' ' v!" T^*' ^P^osmg relationships between print media, the 

weo, and the Netpage Network. 
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Figure 68. Re-purposing content and applications 

10.2 Re-purposing Editorial Content 

Print publishers have been re-purposing content for the Web in vanous ad hoc wavs 
almost since the inception of the Web. In more recent times, tools have be^un to emerge 

h,, r ii ? UbhsherS t0 de P lo >' content in muk 'P le ^edia from a s.ngle Editorial data- 
base [4] The key to such multi-purposing is the separation of content 
tent lavout. 



semantics from con- 



XML ls beginning to be widely used as a markup language to specifV content semantics 
uunng or after story creation, the journalist embeds semantic tags in the text Once 
tagged, the story can be inserted into a page composition specific to a particular medium. 
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cueS co tfj" 1 ^ T b ' "^JV 5 ' aid ° Ut aCC ° rding t0 3 medium-specific stvleshee, 

cued to the semantic tags embedded in the text. 

ll2ZZn Um T dCPe l d T editing PagC com P° sitio " ^s'ems are now emerging 
bach supports output to both pnnt and the Web, and can be extended in a modular fashion 

ns. ght [10], and Worldweb.net's Expressroom [11,12] are notable examples These sys 
nr triT 7' tlp,e - media P^lication into the core editoria. activities of the pub. sh 
<ng staff, rather than treating non-print media as separate and incidental. 

10.3 Re-purposing Web Applications 

oTr'aw HTTP^rrT'T ^^P™" 1 haS imp ™ ed enormously since the early days 
mterfLsiyL h CG ' t ApP ' ,Catlon se ™ rs "™ P^ide high-performance back-end 
mterfaces to database systems, transacts support with commit and rollback and isolate 
the application user mterface [.,6,7,8]. Very little of the effort involved in devllZ t 
muuem weo application goes into Web-specific programming, and as a consequenceThe 
cost of re-purpos-ng a Web application on the Netpage Network is not great The function 

liH , C aPP 'r ti0n rCmainS UnChangCd ' and ° n,y the detailed ""interface flow and 
individual page designs need to change. 

mTs^on!^ T bUrCaU C ° mmerCe ap P' ications are em -ging as well. iCat 
PJ s one of the better-known examples. It offers customized e-commerce hostine to an V 
small company w„h a product catalog. By simply expanding the iCat system to treat "he 
Netpage Network as an additional interface in parallel with the Web, all K*Thost« I busi 
nesses immediately gain entry to the Netpage Network with no effort on their pan. 

10.4 Re-purposing Web Content 

At a more mundane level, the Netpage Network provides an alternative interface to the 
web itself, providing access to entire Web sites in printed form, with several Web paees 

ToTh, ™ e ?t h Ph , ySiCa ' Page ' WCb Pages 3re 3ppear in prin,ed forrn y« ^main f""v nav- 

t ] IZ h° m' 6 War< ;" WHiCh constitutes much of the w <* is often more comfort- 

amy consumed in Netpage form. 

Documents in Adobe PDF format, increasingly prevalent on the Web as companies strive 
o ; r control over hnal pnnt quality and shy away from re-coding information in inferior 
1 lVU " arc even better candidates for Netpage delivery. 
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Overview 



Netpage Printers are intended for use in domestic, commercial, corporate and hospitality 
environments. They ail utilize pagewidth Memjet pnntheads [4], and are based on a sinv 
ple and reliable straight paper path, passing through a Memjet transfer roller printhead 
mechanism. In most cases the printed page is glued along one edge and adhered to the pre- 
vious page to form a final bound document that can be 1 page or 500 pases thick. They all 
interact with the wireless Netpage Pen. 

Netpage Printers come in various forms: wall-mounting, tabletop, portable, and pocket 
versions. 
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2 Wallprinter 

2.1 Wallprinter 

Pan „ • . ' 0, L US Lette r paper in a vert.cal format as shown in Fieures I 3 and 4 

guTdes brS^nr, g ; d p ,OP ^ d ° Wn ° m ° 3 SP ™ g ^ ™« -gisfered u'ni ^ 
gu.des before be.ng closed. F.gure 2 shows the access to the paper andlnk cartridge. 

a^«c C ^ Cya "' ma Sen.a, yellow, and -nfrared inks and g, ue is 

rlulr^Ztw wh aSe -r C r h h r a " OWS P ' aten t0 PUSh the ^ gainst the pick-up 

h e she ' a se T™ e ^ ^ f dUP ' eX ^ Fr0m 
it has Xe ann ied to rh. m °™T7 ^ g ' Ue Whed With pOWered s P ike -heels, where 

not i^^XT^^^ T " g - — * — -n 

^system « ^ d ° Wn '° 3 binding P ' aten ° PerateS with a closed «eel wire 
push rZ fc "7^ " 3 P ° W£red 3Xle - THiS Pr ° videS the necessa ^ speed to 

k " s z , at,;; v : rear °j 3 previous shm - g,ue/bind * - d ^ *> ^ 

Hastic a f d i °u "T iS b T d ^ finished ' a P° wered exit h «ch with a tamper sensor opens 

c i n t:vtdTed : w, ; h f -r h to d,rect the nnished d ° cument <° * e 

TheToMec ^ tr " , m ^ d ° CUmemS ^ the ^ w ith ° Ut hi "'^ ™'*in* ones, 
lain .0 d n A ces f7 7 P 0, - vcarbonate and P" lls ou. of its socket under a cer- 

n °- AccciS ,or amoving documents is provided on three sides. 

™£H72 he's" T a C Vcn aCCOmm ° dateS a " ma J or c^try components include 

- J V P' mcr ^ lJ PP«> 'hat ,s connected before uall-moumint:. 
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Figure 2. Wallprinter paper and ink cartridge access 

The printer has several metal hangers on the rear, which locate into keshole ■ 
I" b3Ck is — , y fastened to a wa.L When thc rh L t tco ZlaJ 

■« .s hung onto the back plate and fixed with a locking screw found under the paper "ay 

c^7^^^ «< co.or.-s the front m0 ,d,n g s 
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2.2 Wallprinter Pro 



This printer is similar to Wallprinter in most respects, except that it has a duplex I 1" print- 
head assembly, which prints on US Letter paper in a landscape format (see Figures 5, 6 
and 7). This means a faster print time and binding time for each page, making for faster 
overall document delivery. 

Another difference is the location of the ink cartridge, which resides above the paper tray 
rather than down the side. Each page is glued along the horizontal edge by a full-length 
glue sponge, which is capped when not in use. Operation, printing, and document han- 
dling are identical to Wallprinter. 

Wallprinter Pro is fully customizable in finishes and color. 




Figure 5. Wallprinter Pro 
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2.3 Wallprinter Pro R 



printer shares the same printing and binding configuration of US fetter landscape 
format as Wallprinter Pro. The main difference is the media delivery, which is in the form 
of a large print cartridge cartridge (see figures 8, 9 and 10). This cartridge accommodates 
C . M, V and infrared inks and glue as well as a 1000 sheet capacity roll of paper. The car- 
tridge can be recharged at nominated outlets when required and it is protected from forger- 
ies b\ an authentication chip [1,2]. 

I he printer has integral structural metalwork to support its weight and a ball bearing track 
tor easy loading and removal 



Wallprinter Pro R is fully customizable in finishes and color. 



Silverbrook Research 



Netpage Printer Design Description 



draft vO 3 





Frinted 

Jocument 




Figure 9. Wallpnnter Pro R front elevation 

7 September 1990 



12 



Silverbrook Research 



Netpage Printe r Design Descnpt,< 



draft vO 3 



Print roll 



Ink 

bladdere 



Kear moldi 



Cover 

molding 

(clip-on) 



Oocu men t 
tray 



&ackp\ate 
(metal) 



Ball bearing 
assembly 




Flat-en motor 
& drum 



Figure 10. Wallprinter Pro R side elevati 



Silverbrook Research 



draft vO 3 



3 Tableprinter 



3.1 Tableprinter Pro 

Ihib printer is a tabletop \etMon of the Uallprinter Pro. Essentially, it is the same printer 
unit with a base plinth that adds extra functionality such as l"SR. parallel port and a power 
socket (see Eigure II). 

lablepnnter Pro is fully customizable in finishes and color. 




Figure 1E Tableprinter Pro 
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3.2 Tableprinter Pro R 



I his printer is a tabletop version of the Wallprinter Pro R l-ssentiallv. it is the same printer 
um« vyth a base plinth that adds extra functionality, such as USB. parallel port and a pow 
MKKet (see figure 12). 

Tableprinter Pro R is fully customizable in finishes and color. 
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4 Deskprinter 



4.1 Deskprinter Pro R 

The Deskprinter Pro R is a vertical format printer based around the large format print car- 
tridge (see Figures 13, 14, 15 and 1 6). This cartridge accommodates C, M. Y, Infrared inks 
and glue, as well as a 1000 page roll of paper. The print cartridge is lowered into the top of 
the unit through a latching hatch. When printing, the paper is fed through the printhead 
assembly where it is duplex printed and then ejected past a cutter which crops it into 
sheets. As the sheet passes the cutter, a glue wheel assembK glues along the lone edge and 
powered spike rollers propel the sheet out into the binding area. 

The binding area consists of a binding platen which operates w ith a closed steel wire loop 
system of pulleys, runners and a powered axle. This prov ides the necessary speed to push 
the glued sheet forward onto the rear of a previous sheet, glue/bind it and return to the 
home position to accept the next printed sheet in less than 2 seconds. A motorized paper 
tapper assembK aligns the sheets in a simultaneous operation. 

When the document has been bound, a series of metal fingers at the top and bottom of the 
document rotate out of the way and the document is pushed into the out tray area. A pow- 
ered hatch opens and a motorized ejector assembly pushes the document through the exit 

Slot O^fh^ unir Tkp Kqt^K fV^n ^l^r-.^c ^ ^ » , , fU,. A^~. r_ i ^ i 

. 11V nKii ^io^ls Ouiu me uuv-ument rcdu\ ior removal. Subsequent 

documents are stacked up in a similar fashion. The user interface is identical to the other 
printers in the range. A large PCB contains all the processing and external interface com- 
ponents and connects to an internal 1 I0V/220V power supply unit. 

Deskprinter Pro R is fully customizable in finishes and color. 






1 



Figure 13. Deskprinter Pro R 
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igure 14. Deskprinter Pro R, with print cartridge extracted 
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F'arer J nve Document Document Fnnted 

roller ejection assembly collection tray document 



Figure 15. Deskprinter Pro R plan 




Figure 16. Deskprinter Pro R elevation 
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5 Travelprinter 



5.1 Travelprinter R 



The Travelprinter R is a small, lightweight, versatile and completely portable Netpage 
Printer (see Figures 17, 19, 20 and 21). It has in-built mobile network communication 
hardware and software, allowing it to download documents anywhere. Travelprinter also 
has communications ports for computer interface printing when required. 

The printer consists of a front and rear molding with a chassis to accommodate the major 
components including a lithium battery and an V/2" duplex Memjet printhead assembly. 

A compact print cartridge cartridge with C, M, Y and infrared inks and paper is used in the 
printer, providing 50 US Letter pages or 100 AS pages. It is protected from forgeries by an 
authentication chip. 




Figure 17. Travelprinter R 

A motorized guillotine assembly cuts the media between the cartridge and the printhead 
and motorized spike wheels eject the finished print out of the unit. A flex PC B runs from 
the main hoard to a segment LCI) and two push buttons 'I he LCI) shows signal strength. 
;m> error v bailer, and number of pages left in the cartridge I he buttons allow the printer 
to either connect to the Netpage Network or to act as a stand-alone printer A I SB inter- 
lace is provided on the side of the printer along w ith DC 3 V input. 
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- Travelprinter R detail, showing duplexed imaging units and print cartridge 
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6 Microprinter 

6.1 Microprinter 

The Microprinter is a small, versatile, pocket-size printer camera (see Figures 22 and 23). 
It has built-in mobile network communication hardware and software, allowing it to link 
to the Netpage Network and fetch documents from anywhere. In addition, the product is 
economically configured and styled as a fully functional digital camera with the standard 
Classic, HDTV and Panoramic print formats for photograph). 

The Microprinter accommodates a 4" (I 00mm) page-width print cartridge with C, M, Y 
and infrared ink plus 5 4 meters of paper. This means the user can single-sided print 41 
Netpages, 36 Classic, 30 Horizontal or 18 Panoramic prints or any combination thereof 




Figure 22. Microprinter rear view with printing in progress (left), 
and front view with print cartridge extracted (right) 

I he Microprinter is a fixed-lens camera with auto focus and dieita! photo enhancement 

capabilities, which allows the u>er to take sophisticated phntns. wr\ MmpK and easiK A 
pr mt buttM- .iM. < P m ■■• r. , r ... j r , ■ ■ ■ * .... 
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head cupping mechanism, a cutter assembK. the roll motor drive. \ anous control switches 
and a CR2 hatter, 

I he Microprinter is fully custom i/ahle in finishes and color. 




Figure 23. Microprinter exploded view 



24 



Silverbrook Research M P)nanp p ri ni^ r 

Netpage Printe r Design Description draft vO 3 



Printer 
Controller 



ilverbrook Research 



Netpage Printer Design Description 



draft vO 3 



Memjet-Based Printing 



A Memjet printhead produces 1600 dpi bi-level CMYK. On low-diffusion paper, each 
ejected drop forms an almost perfectly circular 22.5u;m diameter dot. Dots are easily pro- 
duced in isolation, allowing dispersed-dot dithering to be exploited to its fullest. Since the 
Memjet printhead is the width of the page and operates with a constant paper velocity, the 
four color planes are printed in perfect registration, allowing ideal dot-on-dot printing. 
Since there is consequently no spatial interaction between color planes, the same dither 
matrix is used for each color plane. Dot-on-dot printing minimizes 'muddying 1 of mid- 
tones caused by inter-color bleed. 

A page layout may contain a mixture of images, graphics and text. Continuous-tone (con- 
tone) images and graphics are reproduced using a stochastic dispersed-dot dither. Unlike a 
clustered-dot (or amplitude-modulated) dither, a dispersed-dot (or frequency-modulated) 
dither reproduces high spatial frequencies (i.e. image detail) almost to the limits of the dot 
resolution, while simultaneously reproducing lower spatial frequencies to their full color 
depth, when spatially integrated by the eye. A stochastic dither matrix is carefully 
designed to be free of objectionable low-frequency patterns when tiled across the imase. 
As such its size typically exceeds the minimum size required to support a particular num- 
ber of intensity levels (e.g. 16x16x8 bits for 257 intensity levels). 

Human contrast sensitivity peaks at a spatial frequency of about 3 cycles per decree of 
visual field and then falls off logarithmically, decreasing by a factor of 100 beyond about 
40 c>cles per degree and becoming immeasurable beyond 60 cycles per decree [7,8]. At a 
normal viewing distance of 12 inches (about 300mm), this translates roughly to 200-300 
cycles per inch (cpi) on the printed page, or 400-600 samples per inch according to 
Nyquisfs theorem. 

In practice, contone resolution above about 300 ppi is of limited utility outside special 
applications such as medical imaging. Offset printing of magazines, for example, uses 
contone resolutions in the range 150 to 300 ppi. Higher resolutions contribute slightly to 
color error through the dither. 

Black text and graphics are reproduced directly using bi-level black dots, and are therefore 
not antialiased (i.e. low-pass filtered) before being printed. Text is therefore supersampled 
beyond the perceptual limits discussed above, to produce smoother edges when spatially 
integrated by the eye. Text resolution up to about 1200 dpi continues to contribute to per- 
ceived text sharpness (assuming low-diffusion paper, of course). 

I he Netpage F'rinter uses a contone resolution of 267 ppi (i c 1 6on dpi 6). and a black 
text and graphics resolution of 800 dpi. 
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8 Document Data Flow 



Because of the page-width nature of the Memjet printhead, each pace must be printed at a 
constant speed to avoid creating visible artifacts. This means that the printing speed can't 
be varied to match the input data rate. Document rasterization and document printing are 
therefore decoupled to ensure the printhead has a constant supply of data A pao e is never 
printed until it is fully rastenzed. This is achieved by storing a compressed version of each 
rastenzed page image in memory. 

This decoupling also allows the RIP ,o run ahead of the printer when rastcrmno simple 
pages, buying time to rasterize more complex pages. 

Because contone color images are reproduced by stochastic dithering but black text and 
me graphics are reproduced directly using dots, the compressed page image format con- 
tains a separate foreground bi-level black layer and background contone color laver The 
black layer is composited over the contone layer after the contone layer is dithered. 

Figure 24 shows the now of a Netpage Printer document from network to printed page. 



receive 
document 



/ 

page layouts 
ana objects 




Figure 24. Netpage Printer document datn now 

A: pp.. a 1 etter page of contone Rc ,H or (MY data has a ,:.v ot [ \ m „ 

conuuie compress,, algorithms such as JPLG L^J- contone images compress with "a ratio 
up to 10.1 without noticeable loss of quality, giving a compressed page size of I 9MB 

At 800 dpi. a l etter page of bi-level data has a size of 7MB. Coherent data such as text 
compresses very well Using lossless bi-level compression algorithms such as Group 4 
a-,m,le [(,]. ten -point text compresses w ith a rat,,, of about 10 I (as discussed in Section 
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The two-layer compressed page image format therefore exploits the relative strengths of 
lossy JPEG contone image compression and lossless bi-ievel text compression. The for- 
mat is compact enough to be storage-efficient, and simple enough to allow straightforward 
real-time expansion during printing. 

Since text and images normally don't overlap, the normal worst-case page image size is 
1.9MB (i.e. image only), while the normal best-case page image size is 0.8MB (i.e. text 
only). The absolute worst-case page image size is 2.7MB (i.e. text over image). Assuming 
a quarter of an average page contains images, the average page image size is 1.1 MB. 
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Printer Controller 
Architecture 



The Netpage Printer controller consists of a controlling processor, a factory-selected net 
work interface, a radio transceiver, dual raster image processor (RIP) DSPs duplexed 
print engines, flash memory, and 64MB of DRAM, as illustrated in Figure 25 



Internet 




Figure 25. Basic printer controller architecture 



The controlling processor handles communication with the Internet and with local wire- 
less pens, controls the user interface (buttons and LFDs), controls the paper transport, han- 
dles ink cartridge authentication and ink monitoring and feeds and synchronizes the RIP 
DM's and print entries. It consists of a medium-performance ueneral-purpose micronro- 
cessor. 

I he RIP DSPs rastenze and compress page descriptions to the Netpage Printer's com- 
pressed page format. Each print engine expands, dithers and prints pace images to its asso- 
ciated Memjet printhead in real time (i.e. at 30 or 45 pages per minute). The duplexed 
print engines print both sides of the page simultaneously 

The printer controller's flash memory holds the software for both the processor and the 
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MEMS chip are not used in the printer controller. The Internet network interface module is 
separate, since Netpage Printers allow the network connection to be factory-selected. 
Flash memory and the 2x256Mbit (64MB) DRAM is also off-chip. 

Various Internet network interface modules can be supported. Possibilities include a POTS 
modem, a Hybrid Fiber-Coax (HFC) cable modem, an ISDN modem, a DSL modem, a 
satellite transceiver, a current or next-generation cellular telephone transceiver, a wireless 
local loop (WLL) transceiver, etc. A Internet connection may already be available on the 
local network, in which case only a local network connection may be required. 

The printer controller optionally includes a local network connection, to allow the printer 
to be used from a directly-connected workstation or over a local-area network. Possibili- 
ties include IOBase-Tand 100Base-T Ethernet, USB and USB 2.0, IEEE 1394 (Firewire), 
and various emerging home networking standards. 

The radio transceiver communicates in the unlicensed 900MHz band normally used by 
cordless telephones, and uses frequency hopping and collision detection to provide inter- 
ference-free communication. 

9.1 Detailed Document Data Flow 

I he main processor receives and verifies the document's page layouts and page objects by 
Internet pointcast and multicast. It then runs the appropriate RIP software on the DSPs. 

1 he DSPs rasterize each page description and compress the rasterized page image. The 
main processor stores each compressed page image in memory. The simplest way to 
load-balance multiple DSPs is to let each DSP rasterize a separate page. The DSPs can 

always be kept busy since an arbitrary number of rasterized pages can, in general, be 
stored in memory. This strategy can lead to poor DSP utilization, however, when rasteri/- 

ing short documents. 

Watermark regions in the page description are rasterized to a contone-resolution bi-leve! 
bitmap w hich is losslessly compressed to negligible size and which forms part of the com- 
pressed page image. 

I he infrared UR) layer of the printed page contains encoded position tags at a density of 
about 25 per inch. Each tag encodes the page id, tag position, and control bits. Active 
areas and pressure-sensitive areas in the page description are rasterized to tag-resolution 
bi-level bitmaps w hich do not require compression and which form part of the compressed 
page image 

I he main processor passes back -to -back page images to the duplexed print ermines. Each 
print engine stores the compressed page image into its local memory, and starts the page 
expansion and printing pipeline. Page expansion and printing is pipelined because it is 
impractical to store a 1 14MB bi-level CMY + IR page image in memory. 

I he first stage of the pipeline expands the JPEG-compressed contone CM Y layer, expands 
the Group 4 Fax -com pressed bi-level watermark map, and expands the Group 4 Fax -com- 
pressed bi-level black layer, all in parallel. The second stage dithers the contone CMY 
layer using the dither matrix selected by the watermark map, and composites the bi-level 
black layer over the resulting bi-level CMY layer. Since there is no black ink used in the 
Netpage Printer, the black layer is composited with each of C, M and Y In parallel with 
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this, the tag encoder generates and encodes the bi-level IR tag data. The last stage prints 
the bi-level CMY + IR data through the Memjet printhead via the printhead interface. 

9.2 Print Engine Architecture 

The print engine's page expansion and printing pipeline consists of a standard JPEG 
decoder, a standard Group 4 Fax decoder, a custom halftoner/compositor unit, a custom 
position tag encoder, and a custom interface to the Memjet printhead. These are described 
in detail in Section 1 2. 



printer controller bus 



JPEG I 

decoder I 



contone 
CMY 



tone I bi -level | 
FIFO I w'mark FIFO I 





I position tag I 
I encoder I 



print engine 




Figure 26. Print engine architecture 

U hen several print engines are used in unison, such as in a duplexed configuration, they 
are synchronized via a shared line sync signal. Only one print engine, selected via the 
ex tenia 1 master slave pin, generates the line sync signal onto the shared line. 

In the 8' 2 " versions of the Netpage Printer, the two print engines each prints 30 Letter 

pages per minute along the long dimension of the page (11"), giving a line rate of 8.8 kHz 
ar W>nn dp, | n the 1!" Pro wr-.i.ww rK> v,. r[ , ; ., f . [>,;„.... ,\ , , .. , i 
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1 0 Pen Controller Architecture 

The Netpage Pen controller consists of a controlling processor, a radio transceiver, a tilt 
sensor, a nib pressure sensor, a IR image sensor, Hash memory, and 512KB of DRAM, as 
illustrated in Fiaure 27. 




Figure 27. Pen controller architecture 

I he controlling processor captures and decodes IR position tags from the page via the 
image sensor, monitors the tilt and pressure sensors, controls the autofocus voice-coil, 
controls the user interface (tricolor LED), and handles wireless communication with the 
local Netpage Printer. It consists of a medium-performance (-25 MHz RISC) general-pur- 
pose microprocessor. 

Iwo-axis tilt sensing is provided by a two-axis accelerometer. Nib pressure sensing is pro- 
dded by an IR emitter and photodiode pair in conjunction with a reflector coupled with 
the sprung nib. 

The processor, digital transceiver components, 96x96 image sensor, flash memory and 
512KB DRAM are integrated in a single ASIC. The MI-MS and analog RF components, 
accelerometers. and the IR emitter 'photodiode are integrated in a single MFMS chip, also 
used in the Netpage Printer. 

The radio transceiver communicates in the unlicensed 900MHz band normally used by 

cordless telephones, and uses frequency hopping and collision detection to provide inter- 
ference- free communication. 
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11 Print Engine Page Format 

This section describes the format of compressed pages expected by the print engines. 

and' ,hr r in , lage Pr ° CeSS °!i S (RIPS) 8enera,C pagCS in ,his format ' The impressed format 
and the pnnt engmes are designed to allow real-time page expansion during printing 
ensure that pnnfng ,s never interrupted in the middle of a page due to data undent 

The Netpage Printer reproduces black at full dot resolution (1600 dpi), but reproduces 
contone color a. a somewhat lower resolution using halftoning. The page desertion i 
therefore divided mto a black .ayer and a contone layer. The black layer is defined to com 
posite over the contone layer. om 

The black layer consists of a bitmap containing a 1-bit opacity for each pixel This black 
layer matte has a resolution which is an integer factor of the printer's dot resolution The 
h.ghest supported resolution is 1 600 dpi, i.e. the printer's full dot resolution. 

The conione layer consists of a bitmap containing a 24-bit CMY color for each pixel This 
contone .mage has a resolution which is an integer factor of the printer's dot resolution 
The h,ghest supported resolution is 267 ppi, i.e. one-sixth the printer's dot resolution. 

The contone resolution is also typically an integer factor of the black resolution to sim- 
plify calculates m the RIPs. This is not a requirement, however. 

The black layer and the contone layer are both in compressed form for efficient storaoe in 
tne printer s internal memory. * 

11.1 Page Structure 

The Netpage Printer prints with full edge bleed using an 8'/," printhead. It imposes no 
margins and so has a printable page area which corresponds to the size of its paper The 
target page size .s constrained by the printable page area, less the explicit (target) left and 
top margins spec.fied in the page description. These relationships are illustrated below 




target page 
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11.2 Compressed Page Format 

Apart from being implicitly defined in relation to the printable page area, each page 
description is complete and self-contained. There is no data stored separately from the 
page description to which the page description refers. 

The page description consists of a page header which describes the size and resolution of 
the page, followed by one or more page bands which describe the actual page content. 

Table l shows the format of the page header. 



Table 1. Page header format 



field 


format 


description 


signature 


1 6-bit integer 


Page header format signature 


version 


1 6-bit integer 


Page header format version number. 


structure size 


1 6-bit integer 


Size of page header. 


target resolution (dpi) 


16-bit integer 


Resolution of target page. This is always 
1600 for the Netpage Printer 


target page width 


16-bit integer 


Width of target page, in dots 


target page height 


16-bit integer 


Height of target page, in dots 


target left margin 


16-bit integer 


Width of target left margin, in dots 


target top margin 


16-bit integer 


Height of target top margin, in dots 


black scale factor 


16-bit integer 


Scale factor from black resolution to target 
resolution (must be 2 or greater) 


black page width 


16-bit integer 


Wdth of black page, in black pixels 


black page height 


16-bit integer 


Height of black page, in black pixels 


contone scale factor 


16-bit integer 


Scale factor from contone resolution to tar- 
get resolution (must be 6 or greater) 


contone page width 


16-bit integer 


Width of contone page, in contone pixels 


contone page height 


16-bit integer 


Height of contone page, in contone pixels 



The page header contains a signature and version which allow the print engine to identify 
the page header format. If the signature and or version are missing or incompatible with 
the print engine, then the print engine can reject the page. 



The page header defines the resolution and size of the target page. The black and contone 
layers are clipped to the target page if necessary. This happens whenever the black or con- 
tone scale factors are not factors of the target page w idth or height. 

The target left and top margins define the positioning of the target page uithin the print- 
able page area. 

The black layer parameters define the pixel size of the black layer, and its integer scale 
factor to the target resolution. 

The contone layer parameters define the pixel size of the contone laver, and its integer 
scale factor to the target resolution. 

Table 2 show s the format of the page band header 
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Table 2. Page band header format 



field ,;;/ 




*> '** ■ desc rf Dtio rt >~ * ^i^f r~r- 


signature 


16-bit integer 


Page band header format signature. 


version 


1 fi-hit intADfir 


rdyc Udl 1U M t? dUtr i IUI 1 1 )dl VClolOll N U Ni DC T 


structure size 


16-bit integer 


Size of page band header. 


black band height 


16-bit integer 


Height of black band, in black pixels 


black band data size 


32-bit integer 


Size of black band data, in bytes 


contone band height 


16-bit integer 


Height of contone band, in contone pixels 


contone band data size 


32-bit integer 


Size of contone band data, in bytes 


watermark map band data size 


32-bit integer 


Size of watermark map band data, in bytes 


tag control band height 


1 6-bit integer 


Height of position tag control band, in tags 


active area map band size 


32-bit integer 


Size of active area map band data, in bytes 


pressure-sensitive area map 
band SiZe 


32-bit integer 


Size of pressure-sensitive area band data, 
in bytes 



The black layer parameters define the height of the black band, and the size of its com- 
pressed band data. The variable-size black data follows the page band header. 



The contone layer parameters define the height of the contone band, and the size of its 
compressed page data, consisting of the contone color data and the associated bi-level 
watermark map. The variable-size contone data follows the black data. The variable-size 
bi-level watermark map data follows the contone data. 

I he tag layer parameters define the height of the position tag control band, and the size of 
its active area and pressure-sensitive area map band data. The variable-size active area 
map data follows the watermark map data. The variable-size pressure-sensitive area map 
data follows the active area map data. 

Table 3 show s the format of the variable-size compressed band data which follows the 
page band header. 



Table 3. Page band data format 



field 


format 


description 


h'ack data 


G4Fax 

by te stream 


Compressed bi-level black data 


"■ "t- ' d :t'.-. 


JPEG 

by testream 


... 

Compressed contone CW data 


watermark map 


G4Fax 
bytestream 


Compressed bi-level watermark map data 


active area map 


bitmap 


Bt-level active area map data 


pressure-sensitive area map 


bitmap 


Bi-level pressure-sensitive area map data 



1 a*Ji \ ariable-si/e segment ot band data is aligned to an S-bvTe boundary 
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11.2.1 Bi-level Data Compression 

11.2.1.1 Group 3 and 4 Facsimile Compression 

The Group 3 Facsimile compression algorithm [12] losslessly compresses bi-level data for 
transmission over slow and noisy telephone lines. The bi-level data represents scanned 
black text and graphics on a white background, and the algorithm is tuned for this class of 
images (it is explicitly not tuned, for example, for halftoned bi-level images). The ID 
Group 3 algorithm runlength-encodes each scanline and then Huffman-encodes the result- 
ing runlengths. Runlengths in the range 0 to 63 are coded with terminating codes. Run- 
lengths in the range 64 to 2623 are coded with make-up codes, each representing a 
multiple of 64, followed by a terminating code. Runlengths exceeding 2623 are coded 
with multiple make-up codes followed by a terminating code. The Huffman tables are 
fixed, but are separately tuned for black and white runs (except for make-up codes above 
1 728, which are common). When possible, the 2D Group 3 algorithm encodes a scanline 
as a set of short edge deltas (0, ±1, ±2, ±3) with reference to the previous scanline. The 
delta symbols are entropy-encoded (so that the zero delta symbol is only one bit long etc.) 
Edges within a 2D-encoded line which can't be delta-encoded are runlength-encoded, and 
are identified by a prefix. ID- and 2D-encoded lines are marked differently. ID-encoded 
lines are generated at regular intervals, whether actually required or not. to ensure that the 
decoder can recover from line noise with minimal image degradation. 2D Group 3 
achie\es compression ratios of up to 6: 1 [12]. 

The Group 4 Facsimile algorithm [6] losslessly compresses bi-level data for transmission 
over error-free communications lines (i.e. the lines are truly error-free, or error-correction 
is done at a lower protocol level). The Group 4 algorithm is based on the 2D Group 3 algo- 
rithm, with the essential modification that since transmission is assumed to be error-free, 

1 D-encoded lines are no longer generated at regular intervals as an aid to error-recovery. 
Group 4 achie\e . compression ratios ranging from 20:1 to 60:1 for the CCITT set of test 
images [12). 

The design goals and performance of the Group 4 compression algorithm qualify it as a 

compression algorithm for the bi-level layers. However, its Huffman tables are tuned to a 
lower scanning resolution (100-400 dpi), and it encodes runlengths exceeding 2623 aw k- 
wardly At 800 dpi, our maximum runlength is currently 6400. Although a Group 4 
decoder core might be available for use in the printer controller chip, it might not handle 
runlengths exceeding those normally encountered in 400 dpi facsimile applications, and so 
would require modification. 

11.2.1.2 Edge Delta 'Runlength (EDRL) Compression 

Since most of the benefit of Group 4 comes from the delta-encoding, a simpler algorithm 
based on delta-encoding alone is likeK to meet our requirements. This approach, in the 
form of the Ldge Delta / Runlength (EDRL) encoding, is described in detail in another Sil- 
verbrook design document [3]. There are therefore two viable approaches to losslesslv 
compressing bi-level data: Group 4 Facsimile and EDRL. Their compression performance 
is compared below. Where this document refers to Group 4 Facsimile (G4Fax) compres- 
sion, EDRL can be substituted, purely as an implemetation decision. 

Magazine text is typically typeset in a typeface with serifs (such as l imes) at a point size 
of 10. At this size a Letter page holds up to 14,000 characters, though a typical magazine 
page holds only about 7,000 characters. Text is seldom ty peset at a point size smaller than 
:v At SOU dpi. text cannot be meaningful!) rendered at a point size lower than 2 usin^ a 
standard t> pet ace Table 4 illustrates the legibility of various point sizes 
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Table 4. Text at different point sizes 



Dofnt si7«* 


sample text (In Times) 


2 




3 




4 




5 


"The quick hvtmn lov jump^ the Jo^ 


6 


The quick brown fox jumps o\ct the lazv dog 


7 


TTie quick brown fox jumps over the lazv dot; 


8 


The quick broun fox jumps over the lazy doe 


9 


The quick brown fox jumps over the lazv dog. 


10 


The quick brown fox jumps over the lazy dog. 



Table 5 shows Group 4 and EDRL Compression nerfnrm^n 

point sizes, rendered at 800 dpi. The distribution of charaaers on the test paees is based on 
tnghsh-language statistics [II]. 



Tables. Group 4 and EDRL compression performance on text at 



point size 


characters 
per page 


Group 4 * 
compression 
ratio 


EDRL 

compression 
ratio 


n 


340,000 


2 3 


1.7 


3 


170,000 


3.2 


2.5 


4 


86,000 


4 7 


3.8 


5 


59,000 


5 5 


4.9 


6 


41,000 


6 5 


6.1 


7 


28,000 


7.7 


7.4 


8 


21,000 


9.1 


9.0 


9 


17,000 


10 2 


10 4 


10 


14,000 


10 9 


11 3 


11 


12,000 


11.5 


12 4 


12 


8,900 


13 5 


14 8 


13 


8,200 


13 5 


15 0 


14 


7.000 


14 6 


16 6 


1 5 


5.800 


16 1 


18 5 j 


20 


3.400 


19 8 j 


. ~ 2 ^rij 



800 dpi 



For a point si^e of 9 or greater, FDR[. slightly outperforms Group 4, simplv because 
Group 4 s runlength codes are tuned to 400 dpi. 

These compression results bear out the observation that entropv-encoded runlenoths con- 

tribute much less to compression than 2D encoding unless the data is poorK ennvkited 
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11.2.1.4 Watermark Map Compression Format 

The 267 dpi watermark layer, which matches the 267 ppi contone color layer, is losslessly 
compressed using G4Fa\ (or EDRL) at a typical compression ratio exceeding 50: l . 

11.2.2 Contone Data Compression 

11.2.2.1 JPEG Compression 

The JPEG compression algorithm [9] lossily compresses a contone image at a specified 
quality level. It introduces imperceptible image degradation at compression ratios below 
5: 1, and negligible image degradation at compression ratios below 10: 1 [13]. 

JPEG ty pically first transforms the image into a color space which separates luminance 
and chrominance into separate color channels. This allows the chrominance channels to be 
subsampled without appreciable loss because of the human visual system's relatively 
greater sensitivity to luminance than chrominance. After this first step, each color channel 
is compressed separately. 

The image is divided into 8x8 pixel blocks. Each block is then transformed into the fre- 
quency domain via a discrete cosine transform (DCT). This transformation has the effect 
of concentrating image energy in relatively lower-frequency coefficients, which allows 
higher-frequency coefficients to be more crudely quantized. This quantization is the prin- 
cipal source of compression in JPEG. Further compression is achieved by ordering coeffi- 
cients by frequency to maximize the likelihood of adjacent zero coefficients, and then 
runlength-encoding runs of zeroes. Finally, the runlengths and non-zero frequencv coeffi- 
cients are entropv coded. Decompression is the inverse process of compression. 

11.2.2.2 CMY Contone Layer Compression Format 

The CMY contone layer is compressed to an interleaved color JPEG bvte^iream. The 
interleaving is required for space-efficient decompression in the printer, but may restrict 
the decoder to two sets of Huffman tables rather than four (i.e. one per color channel) [13]. 
If luminance and chrominance are separated, then the luminance channels can share one 
set ot tables, and the chrominance channels the other set. 

It luminance chrominance separation is deemed necessary, either for the purposes of table 
sharing or for chrominance subsampling, then CMY is converted to YCrCb and Cr and Cb 

are duly subsampled. 

Hie JPEG bvte^tream is complete arid self-contained. It contains all data required for 
decompression, including quantization and Huffman tables 
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12 Print Engine 



12.1 Halftoner/Compositor 



The halftoner/compositor unit (HCU) (Figure 30) combines the functions of halftoning the 
contone CMY layer to bi-level CMY, and compositing the black layer over the halftoned 
contone layer. It also selects between the normal and the "watermark" dither matrix on a 
pixel-by-pixef basis, based on the corresponding value in the watermark map. 

The input to the HCU is an expanded 267 ppi CMY contone layer, an expanded ^67 dpi 
watermark map, and an expanded 1 600 dpi black layer. The output from the HCU is a set 
of I 600 dpi bi-level CMY image lines. 

Once started, the HCU proceeds until it detects an end-of-page condition, or until it is 
explicitly stopped via its control register. 

The HCU generates a page of dots of a specified width and length. The width and length 
must be written to the page width and page length registers prior to starting the HCU The 
page width corresponds to the width of the printhead. The pao e length corresponds to the 
length of the target page. 

The HCU generates target page data between specified left and rieht margins relative to 
the page width. The positions of the left and right margins must be written to the left mar- 
gin and right margin registers prior to starting the HCU. The distance from the left margin 
to the right margin corresponds to the target page width. 

The HCU consumes black and contone 'watermark data according to specified black and 
contone page widths. These page widths must be written to the black page width and con- 
tone page width registers prior to starting the HCU. The HCU clips black and contone data 
to the target page width. This allows the black and contone page widths to exceed the tar- 
get page width without requiring any special end-of-line logic at the input FIFO level. 

The relationships between the page width, the black and contone pase widths, and the 
margins are illustrated in Figure 29. 



target left margin 



target page width 



target right margin 



y black page width ] | 

[ ccntone page A:ct:n , 



page A.ctn \of pnnthead) 



Figure 29. Relationships between page widths and margins 
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The MCU scales contone data to printer resolution both horizontally and vertically based 
on a specified scale factor. This scale factor must be written to the contone scale factor 
register prior to starting the HCU. 



Table 6. Halftoner/compositor control and configuration registers 



register ^ 


width *f 




start 


1 


Start the HCU. 


stop 


1 


Stop the HCU. 


page width 


14 


Page width of printed page, in dots This is the number of dots which 
have to be generated for each line 


left margin 


14 


Position of left margin, in dots. 


right margin 


14 


Position of right margin, in dots. 


page length 


15 


Page length of printed page, in dots This is the number of lines 
which have to be generated for each page 


black page width 


14 


Page width of black layer, in dots Used to detect the end of a black 
line 


contone page 
width 


14 


Page width of contone layer, in dots Used to detect the end of a 
contone line 


contone 
scale factor 


4 


Scale factor used to scale contone data to b. -level resolution 



The consumer of the data produced by the HCU is the printhead interface. The printhead 
interface requires bi-level CMY image data in planar format, i.e. with the color planes 
separated. Further, it also requires that even and odd pixels are separated. The output staee 
of the HCU therefore uses 6 parallel pixel FIFOs, one each for even cyan, odd cyan, even 
magenta, odd magenta, even yellow, and odd yellow. 



* 

configuration/control registers 



ciock enable 
geneiator 



— >H dither 



1 



eclor reg 



shift reg 



~ |'e 

contone i i bi-level 
CMY i i watermark 
FIFO ' i FIFO 



shift reg 



rrarqm 



3 shift regTj^ 



3 shift regs -4 



bi level 
K 

FIFO 



3 bi-level 
C. M > 
FIFOs 



3 bi-level 
C. M. Y 
FIFOs 



Figure 30. Halftoner/compositor unit 
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The input contone CM Y FIFO is a full 7KB line buffer. The line is used contone scale fac- 
tor times to effect vertical up-scaling via line replication. FIFO write address wrapping i s 
disabled until the start of the last use of the line. 



12.1.1 Dither 



The dither unit converts a 24-bit contone CMY value into a 3-bit bi-level CMY value. 
Each color component is independently compared with the 8-bit threshold value at the 
current dot position in the dither cell (see Figure 3 l ). 

There are two programmable 64x64 <8-bit dither cells: a normal dither cell and a -water- 
mark^ dither cell. The i-bit watermark value associated with the current contone pixel 
controls the selection of the watermark or normal threshold value. 

The dither cell address generator contains a pair of 6-bit row and column address registers 
Both registers are initialized to zero. The row address is incremented on everv dot line 
advance, modulo 64. The column address is incremented on every dot clock, modulo 64, 



dither cell 
address 
gnerator 



watermark 
dither cell 
64x64x8 



normal 

dither ceil 
64-64 ,8 



comparator 



comparator 



Y 


■A 


comparator 


A 


'e 



12.1.2 Composite 



Figure 31. Dither, with watermarking 



I he composite unit composites a black layer dot over a halttoned CMY la\er dot. If the 
black la>er o P acit> is one, then the halttoned CMY is set to black (i.e. ail ones) I he black 
la\eropacit> is therefore simpK ORcd uith each of the halftoned C. M. and Y value- 



12.1.3 Clock Enable Generator 



The clock enable generator generates enable signals for clocking the contone CMY water- 
mark pixel input, the black dot input, and the CMY dot output 



As described earlier the omHmmc pj\H ip^u* hutv.- 
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I he clock enable generator also generates an even signal which is used to select the even 
or odd set of output dot FIFOs, and a margin signal which is used to generate white dots 
when the current dot position is in the left or right margin of the page. 

I he clock enable generator uses a set of counters. The internal logic of the counters is 
defined in Table 7 The logic of the clock enable signals is defined in Table 8. 



Table 7. Clock enable generator counter logic 



counter 


abbr. 


' w. : 




^cbridWo'n '"fi v 


h*t ^condition > 


dot 


D 


14 


page width 


RP a v EOL b 


(D>0) a elk 


line 


L 


15 


page length 


RP 


(L>0) a EOL 


left margin 


LM 


14 


left margin 


RP . EOL 


(LM>0) elk 


right margin 


RM 


14 


right margin 


RP . EOL 


(RM>0) a elk 


even/odd dot 


E 


1 


0 


RP . EOL 


elk 


black dot 


BD 


14 


black width 


RP . EOL 


(LM=0) a (BD>0) r elk 


contone dot 


CD 


14 


contone width 


RP , EOL 


(LM=0) a (CD>0) a elk 


contone 
sub-pixel 


CSP 


4 


contone scale 
factor 


RP . EOL v (CSP=0) 


(LM=0) a elk 


contone 
sub-line 


CSL 


4 


contone scale 
factor 


RP (CSL=0) 


EOL a elk 



a RP (reset page) condition- external signal 

b EOL (end-of-line) condition (D=0) a (BD=0) a (CD=0) 



Table 8. Clock enable generator output signal logic 



output signal 


condition 


output dct clock enable 


(D>0) a -.EOP 3 


black dot clock enable 


(LM=0) a (BD>0) a -.EOP 


contone pixel clock enable 


(LM=0) a (CD>0) a (CSP=0) a ^EOP 


contone line advance enable 


(CSL=0) - -.EOP 


even 


E=0 


margin 


(LM=0) v (RM=0) 



a EOP (end-of-page) condition: L=0 



12.2 Printhead Interface 

Netpage Printer uses an 8 ; r" CMYK Memjet printhead, as described in Section 14. The 
printhead consist; of 17 segments arranged in 2 segment groups. The first segment group 
contains 9 segments, and the second group contains 8 segments. There are I 3,600 nozzles 
ot each color in the printhead, making a total of 54,400 no/zles. 

I he printhead interface is a standard Memjet printhead interface, as described in Section 
1 5, configured with the following operating parameters 

• MaxColcrs ■- 4 

• SegmentsPerXfer = 9 

• SegmentGroups j 
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Although the printhead interface has a number of external connections, not ali used for an 
8VV printhead, so not all are connected to external pins on the print engine. Specifically, 
the value for SegmentGroups implies that there are only 2 SRCIock pins and 2 SenseSegSe- 
lect pins. All 36 ColorData pins are required, however. 



The position tag encoder encodes the page id of the page being printed, together with the 
current x-y position on the page, into an error-correctably encoded position tag which is 
subsequently printed in infrared ( IR) ink on the page. 



• the page id 

• a pressure sensitive area bitmap at tag resolution 

• an active area bitmap at tag resolution 

It writes a bi-level IR bitstream to the bi-ievel IR F : IFO. The position tag encoder design as 
defined here allows a single page id per printed page, and provides for a paize size of up to 
21.7 inches per dimension (17 >: 4 x 5 12 / 1600). In addition, the position tag encoder 
allows for both landscape and portrait orientations. 

The position tag encoder consists of two stages connected by a buffer. The first stase, the 
tag data encoder, Reed-Solomon encodes the data to be placed in the ta<zs. The second 
stage, the tag formatter^ places the encoded data into the tag format and passes it on to the 
bi-level IR FIFO. Since the tag formatter runs one line of tags behind the tag data encoder, 
the tag data buffer is in fact a double-buffer. The relationship is shown in Fieure 32. 



12.3 



Position Tag Encoder 



The position tag encoder takes the following as input: 
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formation with respect to the dot axis only. Since the macrodot-to-dot ratio is 1 ;4, the posi- 
tion tag encoder operates at 1/4 of the dot frequency (60 MHz). 



12.3.1 Tag Data Buffer 



The tag data buffer consists of two identical buffers. One is read by the tag formatter while 
the other is being written to by the tag data encoder. The two buffers are then logically 

swapped. 

Each buffer contains 256T bits, where T is the maximum number of tags in a given printed 
line of dots. The maximum amount of memory required for each buffer is l6KBytes 
(when T = 5 I 2). To support a l 2-inch printhead, T - 283, so the memory required for each 
buffer is 9056 bytes (55% of the maximum). Note that the remainder of the position ta<* 
encoder does not change in size based on T. 

As the encoded bits are generated by the tag data encoder, they are written to consecutive 
bits in the tag data buffer. The data bits are read out in random-access fashion by the tag 
formatter. Having 256 bits allocated to each tag allows the tag number to be used for gen- 
erating the high bits of the address and the low 8 bits of the address to come from the 
sub-tag address generator. 

The tag data buffer therefore has the structure as shown by Figure 33: 



<4- 
<4- 



196 bits 60 bits 
► 



data bits for tag 0 


< 


data bits for tag 1 




data bits for tag 2 






data bits for tag T- 1 





Figure 33. Tag data buffer logical structure 

12.3.2 Tag Data Encoder 

It is the responsibility of the tag data encoder to produce the 28 bits of encoded mode data 
and 168 bits of encoded tag data for a total of I 96 bits of encoded data. The encoding 
scheme for position tags is detlned in [5]. 

Each tag is represented on the page by a 17x17x4x4 structure of dots. How ever, since the 
tag data formatter is running at only 1 4 of the dot frequency, this results in a time of 1 156 
cycles ( 1 7x 1 7x4 ) per tag. 

I he tag data encoder and control block functions are provided by a simple microprocessor 
core running at 60 Mil/. This is desirable compared to the design effort to implement a 
specific Reed-Solomon encoder for two types of encoding and specific tag codeword con- 
struction. 
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I he 196 bits of encoded tag data are w ritten to one of the tag data buffers while the other 
is being read from by the tag formatter function. 

The coordinates encoded w ithin the tags for a given line of tags w ill depend on the width 
of the printhead and whether or not the pages are being printed in portrait or landscape 
mode, as shown in Figure 34. 



increasing x 



increasing y 





r un i r\Ai i 



LANDSCAPE 



Figure 34. Relationship of coordinates to tags in a tag line 

• If processing a portrait page, each printed line corresponds to changes in \ coordi- 
nates. A printed line of tags therefore has a constant y coordinate, a start x coordinate 
of a and end x coordinate of T-l with a Ax of /. To advance to the next line of tags, the 
y coordinate increases by I and x is reset to 0. 

* If processing a landscape page, each printed line corresponds to changes in y coordi- 
nates. A printed line of tags therefore has a constant x coordinate, a start y coordinate 
of 0. and an end y coordinate of T-l with a Ay of /. To advance to the next'lme of tags, 
the x coordinate is decreased by 1 and v is reset to 0. 

The high level control function for coordinate generation is outlined here in pseudocode. 



For tag Line - 0 to ma xTag Lines 
For tag = C to 1 as t Tag I n L 1 ne 

' : " 7 1 • * pa geld, tag, t. :Pi :>•- 



EnccdeTag [ tag, page Id, tagX, td j! 
" a t a ; X ) 

Lduj : £ 
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• ot the 28 mode bits, 8 bits are directly copied to the output The remaining 20 are gen- 
erated check words based on GF : (16). 

1 herefore the 8 mode bits output and at least 90 bits of the data bits output can be output at 
a rate approximating 1 cycle per bit. The remaining 70 bits must be generated using Reed- 
Solomon encoding. This allows approximately 1000 cycles for 70 bits, or approximately 
14 cycles per encoded bit. Encoding over GF(16) and GF(64) can be accomplished using 
small tables totalling 64 and 256 bytes respectively. 

The bits are stored in the tag data buffer in bit generation order in order to simplify the 
writing process. The reading process (in the tag formatter) has random access to the gener- 
ated bits, but has specific addressing hardware to assist in this task. The order for writing 
bits is as follows: 

• 90-108 bits of data (15-18 >: 6-bit codewords, depending on whether Is or 0s had to be 
inserted to generate non 000000 and non 111111 codewords) based on the 90 bits of data 
(64 bits of pageld, 9 bits of tagX, 9 bits of tagV, 1 bit from the active area bitmap at tag 
resolution, I bit from the pressure sensitive area bitmap at tag resolution and 6 
reserved bits). 

• 78-60 bits of check words (13-10 x 6-bit check words, depending on how manv data 
codewords were written) 

• 8 bits of mode data (2 > 4-bit codewords) 

• 20 bits of check words ( 5 > 4-bit check words) 

12.3.3 Tag Formatter 

The tag formatter is responsible for merging the encoded tag data with the tag structure, 
and placing the dots in the IR buffer in the correct order for printing. The encoded taa data 
is read from the tag data buffer as previously generated b\ the tag data encoder The for- 
matted dots are placed in the bi-level IR buffer such that the same data is clocked in 4 
times. This allows the tag formatter to run at 1/4 of the dot rate. 

We use a simple set of counters for formatting a set of 68 lines of dots (a set of 68 lines 

equates to 17 rows of 4-dot macrodots). The logic for the 6S lines of dots can be repeated 
until the page has finished printing. The within-tag counters index into an address table of 
9-bit entries with the follow ing attributes: 

• If the entry 's high bit =-- 0, then the macrodot comes from an encoded data bit. The 
encoded data bit to use is given by the concatenation of the current tag" ( 10 bits) and 8 
bits of address from the low S-b its of the entrv 

• It the entry 's high bit ~- 1, then the macrodot comes from the constant structure of a 
position tag (the bull"s-e\e and orientation bits) T he entr\ "s low. bit determines the 
state of the macrodot. 

The address table defines the structure of a positional tag at the macrodot level. It there- 
tore has a constant si/e of 289 < 9-bit entries The stored 8-bit addresses simply combine 
the tag encoding structure with the order that the bits are written to the tag data buffer. For 
the constant pan of the tag (the bulks-eve and orientation bits), the entries are simply 
100000000 for a white dot and 100000001 for a black dot 

The tag formatter is shown in pseudocode form: 
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For dotY - 0 to 3 

For tag = 0 to numTags InLine 
For pixel = 0 to 16 

adr = TableLookup[ line, pixel] 
If (adr 5 = 0- 

bit = T.igDataBuffer [tag | adr- 4 i 
Else * 

bit = adr 0 
Endl f 

Place 4 co{)ies of bit in FIFO 
EndFor 
EndFo r 
EndFor 
EndFor 

Swap TagDataBu ff ers 



The pseudocode is readily transferred to logic, as illustrated by Figure 35. 
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Figure 35. Tag formatter 
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13 Print Engine Driver 



I his section discusses rasterizing pages to the internal compressed page format expected 
by the print engine, as defined in Section l l , in terms of the compressed page driver which 
hides this device-dependent behavior from the higher-level raster image processor (RIP). 
The relationship between the RIP, the graphics system, the compressed page driver, and 
the print engine, is illustrated in Figure 36. 




compressed page 




print 
engine 



] 



Figure 36. RIP and print engine driver 

I he RIP and the graphics system are considered generic components and are not defined 
further at this stage. 

13.1 Graphics and Imaging Model 

The compressed page driver is closely coupled with the graphics system used bv the RIP. 
so that the driver can provide device-specific handling for different graphics and imasiriii 
operations, in particular compositing operations and text operations. 

The graphics system renders images and graphics to a nominal resolution specified by the 
compressed page driver, but allows the compressed page driver to take control of render- 
ing text. In particular, the graphics system provides sufficient information to the com- 
pressed page driver to allow it to render and position text at a higher resolution than the 
nominal device resolution. 

The host graphics system requires random access to a contone page buffer at the nominal 

dev ice resolution, into which it composites graphics and imaging objects, but it allows the 
compressed page driver to take control of the actual compositing - i.e. the compressed 
page driver manages the page buffer. 

13.2 Two-Layer Page Buffer 

The compressed page format contains a 267 ppi contone layer and an 800 dpi black layer. 
The black layer is conceptually above the contone layer, i.e. the black laver is composited 
over the contone layer by the print engine. The compressed page driver therefore mam- 
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tains a page buffer which correspondingly contains a medium-resolution contone layer 
and a high-resolution black layer. 

The graphics systems renders and composites objects into the page buffer bottom-up - i e 
later objects obscure earlier objects. This works naturally when there is only a single layer! 
but not when there are two layers which will be composited later. It is therefore necessary' 
to detect when an object being placed on the contone layer obscures something on the 
black layer. to 

When obscuration is detected, the obscured black pixels are composited with the contone 
layer and removed from the black layer. The obscuring object is then laid down on the 
contone layer, possibly interacting with the black pixels in some way. If the compositing 
mode of the obscuring object is such that no interaction with the background is possible" 
then the black pixels can simply be discarded without being composited with the contone 
layer. In practice, of course, there is little interaction between the contone layer and the 
black layer, since images and text rarely overlap. 

The compressed pase driver specifies a nominal map rp^nlntirm nf?A7 ™; ^ 
system. Where possible the compressed page driver relies on the graphics svstem to render 
image and graphics objects to the pixel level at 267 ppi, with the exception of black text. 
The compressed page driver fields all text rendering requests, detects and renders black 
text at 800 dpi, but returns non-black text rendering requests to the graphics system for 
rendering at 267 ppi. 

Ideally the graphics system and the compressed page driver manipulate color in 
device-independent RGB, deferring conversion to device-specific CM Y until the pa^e is 
complete and ready to be sent to the printer. This reduces page buffer requirement/and 
makes compositing more rational. Compositing in in a device-dependent color space is not 
ideal. 

Ultimately the graphics system asks the compressed page driver to composite each ren- 
dered object into the compressed page driver's page buffer. Each such object uses 24-bit 
contone RGB, and has an explicit (or implicitly opaque) opacity channel. 



page buffer 



medium-res 




contone layer 




(24 -bit RGB) 




■m-g-^Jv.nxi,. maa 

1 / x 1 r x - pp, : 





medium-res 

black layer 
<8 - bit opacity) 



24 - 1 it RGB = 19 1MB x 8 -bit opacity 



■■' PP<' 
- 6 4MB 




high-res 

black layer 
( 1 -bit opacty) 



x 1 bit opacity - 7 2MB 



Figure 37. Two-layer page buffer 

Ihe compressed page driver maintains the two-layer page buffer in three parts. The first 
part is the medium-resolution (267 ppi) contone layer. This consists of a 24-bit RGB bit- 
map Ihe second p.irt is ;i medium -resolution hlaek I >\ ••• 'H- ■. >m ■ ; r , 9 i • , 
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value is obtained by averaging the corresponding n x n high-resolution opacity values. 
This corresponds to box-filtered subsampling. The subsamplmg of the black pixels effec- 
tively antialiases edges in the high-resolution black layer, thereby reducing ringing arti- 
facts when the contone layer is subsequently JPEG-compressed and decompressed. 

The structure and size of the page buffer is illustrated in Figure 37. 

13.3 Compositing Model 



For the purposes of discussing the page buffer compositing model, we define the follow- 
ing variables. 

Table 9. Compositing variables 



variable 


description : 


resolution 


format 


n 


medium to high resolution scale factor 






c c 


contone layer color 


medium 


8-bit color component 


c G 


contone object color 


medium 


8-bit color component 


a G 


contone object opacity 


medium 


8-bit opacity 


a L 


medium-resolution black layer opacity 


medium 


8-bit opacity 


ct B 


black layer opacity 


high 


1 -bit opacity 


a r 


black object opacity 


high 


1 -bit opacity 



When a black object of opacity <x r is composited with the black layer, the black layer is 
updated as follows: 



n - I n - 1 

n 

The object opacity is simply ored with the black layer opacity (Fq. 1 ), and the correspond- 
ing part of the medium-resolution black layer is re-computed from the high-resolution 
black layer (Fq. 2) 

When a contone object of color (V and opacity u, ; is composited with the contone laver. 
the contone laver and the black layer are updated as follows: 

(\ = (\ ( 1 - <>-,. ) if ( « 0 - > °) < 
u i ~ 0 it ( a. (i > 0) ( 4 ) 

a fi = 0 if (a, ; >0) (5) 

C c = C Ci ( 1 - a 0 - ) + i a 0 - (6) 

Wherever the contone object hides the black la>er, even if not full\ opaquely the atTected 
black laver pixels are composited with the contone Fiver (f q 3) and removed from the 
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black layer (Eq. 4 and Eq. 5). The contone object is then composited with the contone 
layer (Eq. 6V 



layer (Eq. 6). 

13.4 Page Compression 



Once page rendering is complete, the compressed page driver converts the contone layer 
to printer-specific CMY with the help of color management functions provided by the 
graphics system. 

The compressed page driver then compresses and packages the black layer and the con- 
tone layer into the compressed page format defined in Section ll. This is subsequently 
expanded in real time by the print engine. 

The forward discrete cosine transform (DCT) is the costliest part of JPEG compression. In 
current high-quality software implementations, the forward DCT of each 8x8 block 
requires 12 integer multiplications and 32 integer additions [10]. This places a trivial addi- 
tional load on each RIP DSP. 
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Memjet Printhead 
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14 Memjet Printhead 

A Memjet printhead is a drop-on-demand 1600 dpi Inkjet printer that produces bi-leve! 
dots in up to 4 colors to produce a printed page of a particular width. Since the printhead 
prints dots at 1600 dpi, each dot is approximately 22.5u.rn in diameter, and spaced 
1 5.875pm apart. Because the printing is bi-level, the input image should be dithered or 
error-diffused for best results. 




Figure 38. A Memjet printhead 

A Memjet printhead is composed of a number of identical 12 inch Memjet segments. The 
segment is therefore the basic building block for constructing a printhead 

14.1 The Structure of a Memjet Segment 

This section examines the structure of a single segment, the basic building block for con- 
structing Memjet printheads. 

14.1.1 Grouping of Nozzles Within a Segment 

I he nozzles within a single segment are grouped for reasons of physical stability as well 
as minimization of power consumption during printing in terms of ph\sical stability a 
total of 10 nozzles share the same ink reservoir. In terms of power consumption, uroupmiis 
are made to enable a low -speed and a high-speed printing mode 

Memjet segments support two printing speeds to allow speed power consumption 
trade-ofTs to be made in different product configurations. 

In the low-speed printing mode, 4 nozzles of each color are tired from the segment at a 
time. The exact number of nozzles fired depends on how many colors are present in the 
printhead. In a four color (e g CMYK) printing environment this equates to I 6 nozzles fir- 
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In the high-speed printing mode, 8 nozzles of each color are fired from the segment at a 
time. The exact number of nozzles fired depends on how many colors are present in the 
printhead. In a four color (e.g. CMYK) printing environment this equates to 32 nozzles fir- 
ing simultaneously. In a three color (e.g. CMY) printing environment this equates to 24 
nozzles firing simultaneously. To fire all the nozzles in a segment, 100 different sets of 
nozzles must be fired. 

The power consumption in the low -speed mode is half that of the high-speed mode. Note, 
however, that the energy consumed to print a page is the same in both cases. 

14.1.1.1 10 Nozzles Make a Pod 

A single pod consists of 10 nozzles sharing a common ink reservoir. 5 nozzles are in one 
row, and 5 are in another. Each nozzle produces dots 22.5jJ.rn in diameter spaced on a 
I5.875u.rn grid to print at 1600 dpi. Figure 39 shows the arrangement of a single pod, with 
the nozzles numbered according to the order in which they must be fired. 




Figure 39. A single pod, numbered by firing order 

Although the nozzles are fired in this order, the relationship of nozzles and physical place- 
ment of dots on the printed page is different. The nozzles from one row represent the even 
dots from one line on the page, and the nozzles on the other row represent the odd dots 
from the adjacent line on the page. Figure 40 shows the same pod u ith the nozzles num- 
bered according to the order in which they must be loaded. 




Figure 40. A single pod, numbered by load order 

The nozzles within a pod are therefore logical I > separated by the width of I dot. The exact 
distance between the nozzles will depend on the properties of the Memjet firing mecha- 
nism. The printhead is designed with staggered nozzles designed to match the flow- of 
paper. 

14.1 .1.2 1 Pod of Each Color Makes a Chromapod 

One pod of each color are grouped together into a chromapod The number of pods in a 
chromapod will depend on the particular application. In a monochrome printing system 
(such as one that prints only black), there is only a single color and hence a single pod 
Photo printing application prmtheads require 3 colors (cyan, magenta, yellow), so Memjet 
segments used for these applications will have 3 pods per chromapod (one pod of each 
color). The expected maximum number of pods in a chromapod is 4, as used in a CMYK 
(cyan, magenta, yellow, black) print irm system {such as a desktop printer). This maximum 
of 4 colors is not imposed b\ any phy sical constraints - it is merely an expected maximum 
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from the expected applications (of course, as the number of colors increases the cost of the 
segment increases and the number of these larger segments that can be produced from a 
single silicon wafer decreases). 

A chromapod represents different color components of the same horizontal set of 10 dots 
on different lines. The exact distance between different color pods depends on the Memiet 
operating parameters, and may vary from one Memjet design to another. The distance is 
considered to be a constant number of dot-widths, and must therefore be taken into 
account when printing: the dots printed by the cyan nobles will be for different lines than 
those printed by the magenta, yellow or black nozzles. The printing algorithm must allow 
for a variable distance up to about 8 dot-widths between colors. Figure 4 1 illustrates a sin 
gle chromapod for a CMYK printing application. 




Figure 41. A Single Chromapod Contains 1 Pod of each Color 
14. 1.1.3 5 Chromapods Make a Podgroup 

5 chromapods are organized into a single podgroup. A pod 2 roup therefore contains SO 
nozzles for each color. The arrangement is shown in Figure 42, with chromapods num- 
bered 0-4 and using a CMYK chromapod as the example. Note that the distance between 
adjacent chromapods is exaggerated for clarity. 

0 12 3 4 



5 chromapods 



Figure 42. A Single Podgroup Contains 5 Chromapods 

14. 1.1 4 2 Podgroups Make a Phasegroup 

2 podgroups are organized into a single phasegroup. The phasegroup is so named because 
groups of nozzles within a phasegroup are fired simultaneously durinn a given firm- 
phase (this is explained in more detail below) T he formation of a phasegroup from "> pod" 
groups is entirely for the purposes of lou -speed and high-speed printing via "> Pod- 
groupFnable lines ^ 
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F : igure 43 illustrates the composition of a phasegroup. The distance between adjacent pod- 
groups is exaggerated for clarity. 



Figure 43. A single phasegroup contains 2 podgroups 
14. 1.1.5 2 Phasegroups Make a Firegroup 

Two phasegroups (PhasegroupA and PhasegroupB) are organized into a smzle ftregroup, 
with 4 firegroups in each segment. Firegroups are so named because they alfflre the same 
nozzles simultaneously. Two enable lines, AEnable and BEnable, allow'the firing of Pha- 
segroupA nozzles and PhasegroupB nozzles independently as different firing phases. The 
arrangement is shown in Figure 44. The distance between adjacent groupings is exagger- 
ated for clarity. 




Figure 44. Relationship between Segments, Firegroups, 
Phasegroups, Podgroups and ChromaPods 
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14. 1.1.6 Nozzle Grouping Summary 



14.1.2 



Table 10 is a summary of the nozzle groupings in a segment assuming a CMYK chroma- 



TablejIO. Nozzle Groupings for a single segment 



Name of - J 
Grouping ; > ' 




^Replication 

-^P^Hol^^' 


% Nozzle c 
: - Count ; r 


Nozzle 


Base unit 


1:1 


1 


Pod 


Nozzles per pod 


10 1 


10 


Chromapod 


Pods per chromapod 


C:1 


10C 


Podgroup 


Chromapods per podgroup 


5:1 


50C 


Phasegroup 


Podgroups per phasegroup 


2.1 


100C 


Piregroup 


Phasegroups per firegroup 


2:1 


200C 


Segment 


Firegroups per segment 


4:1 


800C 



The value of C. the nnmhw r.f ™i„-,- . j ■ ., 

ber of nozzles. "" ,u,, " cu "' l " e se §ment. determines the total num- 

• With a 4 color segment, such as CMYK, the number of nozzles per segment is 3 ^00 

• W.th a 3 color segment, such as CM Y, the number of nozzles per segment is 2,400 

• In a monochrome environment, the number of nozzles per segment is 800. 

Load and Print Cycles 

S twt 7?™ zf ms a ; otal °j * ooc nozz,es - where c is the numb - - 

sc ment. A Pnn, Cycle mvolves the firing of up to all of these nozzles dependent on the 

z:: z b pr, :i A Loa t cycie invo,ves the ^ 

mtoimation to be printed during the subsequent Print Cycle. 

wm h n nOZ H lehaS T a „ SS ° Ciated No ^Enable bit that detennines whether or not the nozzle 
set o! !Z^;: rmt Cyde per nozzle) are loaded via a 

a^shifte'dTn^rsh ? 1 " ^"T ^ ^ *** ^ Cach 800 d ^ ^ bits 

n " les on 'T ' ! V eg,S , ter 3 giVCn C0, ° r thCy arC dirCCted to the »d upper 

< i'e, h W ? CS ' ntC T a " y - MCh 8 °°- dcep ShlTt re -« is,cr ls ^"Prised cf wo 
n h , ; . " °" C f ° r UPPCr n ° /zleS ' and ™ «»'«■■ >o«er nozzles Alter- 

Sh " R,d '"^ a " tfn,a,e in '<™' As far as the external interne s 

->-K,rneJ h.maer. there ,s a single 800 deep shift register 

Once all the shift registers have been fully loaded (800 load pulses) all of the bits are 

S :sfr o e app ; opriate «*■ - : :, e ^ 

; t r " C ? ,e ,ransfcr has ,akcn p,ace ' the PrinI C > d - « n bcein T he 

' " ,> U V? nd he [ -«ad Cycle can occur simultaneously as lorn: as the parallel load of all 
Nozzlefcnable bits occurs at the end of the Print Cvcle ' I load of all 
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tfach segment has C inputs directly related to the C shift registers (where C is the number 
of colors in the segment). These inputs are named ColorSData, where N is 1 to C (for 
example, a 4 color segment would have 4 inputs labeled ColorlData, Cotor2Data, 
Color3Data and Color4Data). A single pulse on the SRClock line transfers C bits into the 
appropriate shift registers. Alternate pulses transfer bits to the lower and upper nozzles 
respectively. A total of 800 pulses are required for the complete transfer of data. Once all 
800C bits have been transferred, a single pulse on the PTransfer line causes the parallel 
transfer of data from the shift registers to the appropriate NozzleEnable bits. 

The parallel transfer via a pulse on PTransfer must take place after the Print Cvcle has fin- 
ished. Otherwise the NozzleEnable bits for the line being printed will be incorrect. 

It is important to note that the odd and even dot outputs, although printed during the same 
Print Cycle, do not appear on the same physical output line. The physical separation of 
odd and even nozzles within the printhead, as well as separation between nozzles of differ- 
ent colors ensures that they will produce dots on different lines of the page. This relative 
difference must be accounted for when loading the data into the printhead. The actual dif- 
ference in lines depends on the characteristics of the inkjet mechanism used in the print- 
head. The differences can be defined by variables D. and D : where D, is the distance 
betw een nozzles of different colors, and D 2 is the distance between nozzles of the same 
color. Table 1 1 shows the dots transferred to a C color segment on the first 4 pulses. 

Table 11. Order of Dots Transferred to a Segment 



Pulse 


Dot 


Colorl Line 


Color2 Line 


Coror3 Line 


Colore Line 


1 


0 


N 


N + D, a 


N + 2Dt 




2 


1 


N+D 2 b 


N+D!+D 2 


N+2Dt+D 2 


N+CC-IJDt+Dj 


3 


2 


N 


N + Dt 


N + 2D : 


N + K-IJDt 


4 


3 


N + D 2 


N + D 1 +D 2 




N + CC-IjDt + Do 



b D 2 = number of lines between two rows of nozzles of the same color (likely = 1 ) 
And so on for all 800 pulses. 

Data can be clocked into a segment at a maximum rate of 20 MHz, which will load the 
entire 800C bits of data in 40us. 



14.1.2.2 Print Cycle 



A single Memjet printhead segment contains 800 nozzles. To tire them all at once would 
consume too much power and be problematic in terms of ink refill and nozzle interference. 
This problem is made more apparent when we consider that a Memjet printhead is com- 
posed of multiple I -'2 inch segments, each u ,th 800 nozzles Consequent two firing 
modes are defined: a lou -speed printing mode and a high-speed printing mode: 

• In the low -speed print mode, there are 200 phases, with each phase tiring AC nozzles 
{C per h regroup, vshere C is the number of colors ). 

* In the high-speed print mode, there are 100 phases, with each phase firing 8C nozzles, 
(2C per fl regroup, where C is the number of colors) 

I he no/zles to be tired in a given firing pulse are determined b> 
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• 3 bits ChromapodSelect (select 1 of 5 chromapods from a firegroup) 

• 4 bits NozzleSelect (select 1 of 1 0 nozzles from a pod) 

• 2 bits of PodgroupEnable lines (select 0, 1, or 2 podgroups to fire) 

When one of the PodgroupEnable lines is set, only the specified Podgroup's 4 nozzles will 
tire as determined by ChromapodSelect and NozzleSelect. When both of the Pod-roupEn 
able lines are set, both of the podgroups will fire their nozzles. For the low-speed mode 
two fire p U i S es are required, with PodgroupEnable = 10 and 01 respectively For the 
high-speed mode, only one fire pulse is required, with PodgroupEnable =11. 

The duration of the firing pulse is given by the AEnable and BEnable lines which fire the 
PhasegroupA and PhasegroupB nozzles from all flregroups respectively. The typical dura- 
tion of a firing pulse is 1 .3 - 1 .8 jis. The duration of a pulse depends on the viscosity of the 
ink (dependent on temperature and ink characteristics) and the amount of power available 
to the pnnthead. See Section 14.1.3 on page 61 for details on feedback from the printhead 
in order to compensate for temperature change. 



o m order that the fn ing pulses can overlap. 



The AEnable and RFnahl^ 

I hus the 200 phases of a low-speed Print Cycle consist of 100 A phases and 1 00 B phases 
effectively giving 100 sets of Phase A and Phase B. Likewise, the 100 phases of a 
high-speed pnnt cycle consist of 50 A phases and 50 B phases, effectively Rivin* 50 
phases of phase A and phase B. 

Figure 45 shows the AEnable and BEnable lines during a typical Print Cvcle In a high- 
speed print there are 50 2)is cycles, while in a low-speed print there are 100 2us cycles" 



Nominal 



AEnable 



BEnable 



Nominal 



2 MS 



2 us 



Figure 45. AEnable and 

f-orthe high-speed printing mode, the 

ChromapodSelect 0, No//!eSe!ect 
ChromapodSelect 1, NozzleSelect 
ChromapodSelect 2, NozzleSelect 
ChromapodSelect 3, NozzleSelect 
ChromapodSelect 4, NozzleSelect 
ChromapodSelect 0. NozzleSelect 



BEnable During a Typical Print Cycle 



firing order is: 

0. PodgroupEnable 1 1 (Phases A and B) 
0, PodgroupEnable 1 1 (Phases A and B) 
0. PodgroupEnable 1 1 (Phases A and B) 
0, PodgroupEnable 1 I (Phases A and B) 

0, PodgroupEnable 1 1 (Phases A and B) 

1. PodgroupEnable 1 I (Phases A and B) 
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For the low-speed printing mode, the firing order is similar. For each phase of the high 
speed mode where PodgroupEnable was I 1, two phases of PodgroupEnable = 01 and 10 
are substituted as follows: 

• ChromapodSelect 0, NozzleSelect 0, PodgroupEnable 0 1 (Phases A and B) 

• ChromapodSelect 0, NozzleSelect 0, PodgroupEnable 10 (Phases A and B) 

• ChromapodSelect 1 , NozzleSelect 0, PodgroupEnable 0 1 (Phases A and B ) 

• ChromapodSelect 1, NozzleSelect 0, PodgroupEnable 10 (Phases A and B ) 
• 

• ChromapodSelect 3, NozzleSelect 9, PodgroupEnable 01 (Phases A and B) 

• ChromapodSelect 3, NozzleSelect 9, PodgroupEnable 10 (Phases A and B) 

• ChromapodSelect 4, NozzleSelect 9, PodgroupEnable 0 1 (Phases A and B) 

• ChromapodSelect 4, NozzleSelect 9, PodgroupEnable 10 (Phases A and B) 

When a nozzle fires, it takes approximately 100|as to refill. The nozzle cannot be fired 
before this refill time has elapsed. This limits the fastest printing speed to 1 00u.s per line. 
In the high-speed print mode, the time to print a line is 100|is, so the time between firing a 
nozzle from one line to the next matches the refill time. The low-speed print mode is 
slower than this, so is also acceptable. 

The tiring of a nozzle also causes acoustic perturbations for a limited time within the com- 
mon ink reservoir of that nozzles pod The perturbations can interfere with the firing of 
another nozzle within the same pod. Consequently, the firing of nozzles w ithin a pod 
should be offset from each other as long as possible. We therefore fire four nozzles from a 
chromapod (one nozzle per color) and then move onto the next chromapod within the pod- 
group. 

• In the low-speed printing mode the podgroups are fired separately. Thus the 5 chroma- 
pods w ithin both podgroups must all fire before the first chromapod fires again, total- 
ling 10 x 2|is cycles. Consequently each pod is fired once per 20us. 

• In the high-speed printing mode, the podgroups are tired together. Thus the 5 chroma- 
pods within a single podgroups must all tire before the first chromapod fires again, 
totalling 5 x 2us cycles. Consequently each pod is fired once per 10 us. 

As the ink channel is 300u_m long and the velocity of sound in the ink is around 1 500m/s, 
the resonant frequency of the ink channel is 2.5 MHz. Thus the low-speed mode allows 50 
resonant cycles for the acoustic pulse to dampen, and the high-speed mode allows 25 reso- 
nant cycles. Consequent]) an\ acoustic interference is minimal in both cases. 



nf...Vnt\j! 
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14.1.3 Feedback from a Segment 

A segment produces several lines of feedback. The feedback lines are used to adjust the 
timing of the firing pulses. Since multiple segments are collected together into a printhead 
ins effective to share the feedback lines as a tri-state bus, with onlv one of the segments' 
placing the feedback information on the feedback lines. 

A pulse on the segment's SenseSegSelect line ANDed with data on ColorlData selects if 
the particular segment will provide the feedback. The feedback sense lines will come from 
that segment until the next SenseSegSelect pulse. The feedback sense lines are as follows: 

• Tsense informs the controller how hot the printhead is. This allows the controller to 
adjust timing of firing pulses, since temperature affects the viscosity of the ink. 

• \ sense informs the controller how much voltage is available to the actuator This 
allows the controller to compensate for a flat battery or high voltage source by adjust- 
ing the pulse width. 

• ^s£w« informs the controller of the resistivity (Ohms per square) of the actuator 
heater. This allows the controller to adjust the pulse widths to maintain a constant 
energy irrespective of the heater resistivity. 

• U sense informs the controller of the width of the critical part of the heater, which may 
vary up to ± 5% due to lithographic and etching variations. This allows the controller 
to adjust the pulse width appropriately. 

14.1.4 Preheat Cycle 

The printing process has a strong tendency to stay at the equilibrium temperature To 
ensure that the first section of the printed photograph has a consistent dot size, the equilib- 
rium temperature must be met before printing any dots. This is accomplished via a preheat 
cycle. 

The Preheat cycle involves a single Load Cycle to all nozzles of a segment with Is (i e 
setting all nozzles to fire), and a number of short firing pulses to each nozzle. The duration 
of the pulse must be insufficient to fire the drops, but enouah to heat up the ink. Altoeether 
about 200 pulses for each nozzle are required, cyclina throuah in the same sequence as a 
standard Print Cycle. 

Feedback during the Preheat mode is provided by Tsense, and continues until equilibrium 
temperature ,s reached (about 30" C above ambient). The duration of the Preheat mode is 
around M) milliseconds, and depends on the ink composition 

Preheat is performed before each print job I his docs no, affect performance as ,, ,s done 
while the data is being transferred to the printer 

14.1.5 Cleaning Cycle 

In order to reduce the chances of no/zles becoming closed, a clean, n» evele can be 
undertaken before each print job. Lach nozzle is fired a number of times into an absorbent 

sponge. 
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printer has ben idle. As with preheat, the cleaning cycle has no effect on printer perfor- 
mance. 

14.1.6 Printhead Interface Summary 

Each segment has the following connections to the bond pads: 



Table 12. Segment Interface Connections 



' r. Name 2;^; 


; Unes * 




Chromapod Select 


3 


Select which chromapod will fire (0-4) 


NozzleSelect 


4 


Select which nozzle from the pod will fire (0-9) 


PodgroupEnable 


2 


Enable the podgroups to fire (choice of 01 , 10, 11) 


AEnable 




Firing pulse for podgroup A 


BEnable 




Firing pulse for podgroup B 


ColorNData 


C 


Input to shift registers (1 bit for each of C colors in the segment) 


SRClock 




A pulse on SRClock (ShiflRegisterClock) loads C bits from Color- 
Data into the C shift registers 


PTransfer 




Parallel transfer of data from the shift registers to the internal 
NozzleEnable bits (one per nozzle) 


SenseSegSelect 




A pulse on SenseSegSelect ANDed with data on CoioMData 
selects the sense lines for this segment. 


Tsense 




Temperature sense 


Vsense 




Voltage sense 


Rsense 




Resistivity sense 


Wsense 




Width sense 


Logic GND 




Logic ground 


Logic PWR 




Logic power 


V- 


2 1 | Actuator Ground 


V+ 


21 


Actuator Power 


TOTAL 


62+C 


(:f C is 4, Total = 66) 
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14.2 Making Memjet Printheads out of Segments 

A Memjet printhead is composed of a number of identical i/2 inch printhead segments. 
These 1/2 inch segments are manufactured together or placed together after manufacture 
to produce a printhead of the desired length. Each I/2 inch segments prints 800 1 600 dpi 
bi-level dots in up to 4 colors over a different part of the page to produce the final image. 
Although each segment produces 800 dots of the final image, each dot is represented by a 
combination of colored inks. 

A 4-inch printhead, for example, consists of 8 segments, typically manufactured as a 
monolithic printhead. In a typical 4-color printing application (cyan, magenta, yellow, 
black), each of the segments prints bi-level cyan, magenta, yellow and black dots over a 
different part of the page to produce the final image. The positions of the segments are 
shown in Figure 46. 



1 segment 
= 0.5 inch 
= 12700 jam 




Figure 46. Arrangement of Segments in a 4-inch Printhead 

An 8-inch printhead can be constructed from two 4-inch printheads or from a single 8-inch 
printhead consisting of 1 6 segments. Regardless of the construction mechanism, "the effec- 
tive printhead is still 8 inches in length. 

A 2-inch printhead has a similar arrangement, but only uses 4 segments. Likewise, a 
full-bleed A4 Letter printer uses I 7 segments for an effective 8' V 1 printing area. 

Since the total number of nozzles in a segment is 800C (see Table 10), the total number of 
nozzles in a given printhead with S segments is 800CS. Thus segment N is responsible for 
printing dots 800 N to SOON - 799. 



A number ot considerations must be made when wiring up a printhead As the width of the 
printhead increases, the number of segments increases, and the number of connections 
also increases. Each segment has its own ColorData connections (C of them), as well as 
SRC lock and other connections for loading and printing. 
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14,2.1 Loading Considerations 

When the number of segments S is small it is reasonable to load all the segments simulta- 
neously by using a common SRCIock line and placing C bits of data on each of the Color- 
Data inputs for the segments. In a 4-inch printer, S-8, and therefore the total number of 
bits to transfer to the printhead in a single SRCIock pulse is 32. However for an 8-inch 
printer, S=]6, and it is unlikely to be reasonable to have 64 data lines running from the 
print data generator to the printhead. 

Instead, it is convenient to group a number of segments together for loading purposes. 
Each group of segments is small enough to be loaded simultaneously, and share an 
SRCIock, For example, an 8-inch printhead can have 2 segment groups, each segment 
group containing 8 segments. 32 ColorData lines can be shared for both groups, with 2 
SRCIock lines, one per segment group. 

When the number of segment groups is not easily divisible, it is still convenient to group 
the segments. One example is a 8'/T printer for producing A4 Letter pages. There are 17 
segments, and these can be grouped as two groups of 9 (9C bits of data going to each seg- 
ment, with all 9C bits used in the first group, and only 8C bits used for the second sroup), 
or as 3 groups of 6 (again, C bits are unused in the last group) 

As the number of segment groups increases, the time taken to load the printhead increases. 
When there is only one group, 800 load pulses are required (each pulse transfers C data 
bits). When there are G groups, 800G load pulses are required. The bandwidth of the con- 
nection between the data generator and the printhead must be able to cope and be within 
the allowable timing parameters for the particular application. 

If G is the number of segment groups, and L is the largest number of segments in a group, 

the printhead requires LC ColorData lines and G SRCIock lines. Regardless of G onlv a 
smgle PTransfer line is required - it can be shared across all segments. 

Since L segments in each segment group are loaded with a single SRCIock pulse, any 
printing process must produce the data in the correct sequence for the printhead. As an 
example, when G"2 and L~4, the first SRCIock 1 pulse will transfer the ColorData bits for 
the next Print Cycle's dot 0, 800, 1600, and 2400. The first SRClock2 pulse will transfer 
the ColorData bits for the next Print Cycle's dot 3200, 4000, 4800, and 5600. The second 
SRCIock I pulse will transfer the ColorData bits for the next Print Cycle's dot 1. 801, 
1601, and 2401. The second SRClock2 pulse will transfer the ColorData bits for the next 
Print Cycle's dot 3201, 4001, 4801 and 5601. 

After 800G SRCIock pulses (800 to each of SRCIockl and SRClock2), the entire line has 
been loaded into the printhead, and the common PTransfer pulse can be niven. 

It is important to note that the odd and even color outputs, although printed during the 
same Print Cycle, do not appear on the same physical output line. The phvsical separation 
of odd and even nozzles within the printhead. as well as separation between nozzles of dif- 
ferent colors ensures that they will produce dots on different lines of the pa<ze. This rela- 
tive difference must be accounted for when loading the data into the printhead. The actual 
difference in lines depends on the characteristics of the inkjet mechanism used in the print- 
head The differences can be defined bv variables D, and D : where D is the distance 
between nozzles of different colors, and D : is the distance between nozzles of the same 
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color. Considering only a single segment group, Table 13 shows the dots transferred to 
segment n of a printhead during the first 4 pulses of the shared SRClock. 

Table 13. Order of Dots Transferred to a Segment in a Printhead 



Pulse 


Dot 


Color! Lino \i 


rJColor2 Une 


Colore Une 


1 


800S a 


N 


N+D^ 




2 


800S+1 


N+D 2 C 


N+Dt + D 2 


N+(C-1)D 1 +D 2 


3 


800S+2 


N 


N+D t 


N+(C-1)D! 


4 


800S+3 


N+D 2 


N+D 1 +D 2 


N+(C-1)D 1 +D 2 



b. D, - number of lines between the nozzles of one color and the next (likely = 4-8) 

c. D 2 = number of lines between two rows of nozzles of the same color (likely = 1 ) 

And so on for all 800 SRClock pulses to the particular segment group. 

14.2.2 Printing Considerations 

With regards to printing, we print 4C nozzles from each segment in the low-speed printing 
mode, and 8C nozzles from each segment in the high speed printing mode. 

While it is certainly possible to wire up segments in any way, this document oniv consid- 
ers the situation where all segments fire simultaneously. This is because the low-speed 
printing mode allows low-power printing for small printheads (e.g. 2-inch and 4-inch), 
and the controller chip design assumes there is sufficient power available for the large 
print sizes (such as 8-1 8 inches). It is a simple matter to alter the connections in the print- 
head to allow grouping of firing should a particular application require it. 

When all segments are fired at the same time 4CS nozzles are fired in the low-speed print- 
ing mode and 8CS nozzles are fired in the high-speed printing mode. Since all segments 
print simultaneously, the printing losic is the same as defined in Section 14 1 2 2 on paae 
58. ^ ' ^ 

I he timing for the two printing modes is therefore: 

• 200 us to print a line at low speed (comprised of 100 2|is cycles) 

• 100 Ms to print a line at high speed (comprised of 50 2 us cvcles) 

14.2.3 Feedback Considerations 

A segment produces several lines of feedback, as defined in Section I4.1.3 on pa-e 6l. 
The feedback lines are used to adjust the timing of the firing pulses. Since multiple seg- 
ments are collected together into a printhead, it is effective to share the feedback lines as~a 
tri-state bus, with only one of the segments placing the feedback information on the feed- 
back lines at a time. 



Since the selection of which segment will place the feedback inf.n-m 
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Just as there are G SRClock lines (a single line is shared between segments of the same 
segment group), there are G SenseSegSelect lines shared in the same way. When the cor- 
rect SenseSegSelect line is pulsed, the segment of that group whose ColorlData bit is set 
will start to place data on the shared feedback lines. The segment previously active in 
terms of feedback must also be disabled by having a 0 on its Colorl Data bit, and this seg- 
ment may be in a different segment group. Therefore when there is more than one segment 
group, changing the feedback segment requires two steps: disabling the old segment, and 
enabling the new segment. 

14.2.4 Printhead Connection Summary 

I his section assumes that a printhead has been constructed from a number of segments as 
described in the previous sections. It assumes that for data loadin g purposes, the segments 
have been grouped into G segment groups, with L segments in the largest segment group. 
It assumes there are C colors in the printhead. It assumes that the firing mechanism for the 
printhead is that all segments fire simultaneously, and only one segment at a time places 
feedback information on a common tri-state bus. Assuming all these things, Table 1 4 lists 
the external connections that are available from a printhead: 



Table 14. Printhead Connections 



Name 


#Pins 


Description 


ChromapodSelect 


3 


Select which chromapod will fire (0-4) 


Nozzle Select 


4 


Select which nozzle from the pod will fire (0-9) 


PodgroupEnable 


2 


Enable the podgroups to fire (choice of 01, 10. 11) 


AEnable 


1 


Firing pulse for phasegroup A 


BEnable 


1 


Firing pulse for phasegroup B 


ColorData 


CL 


Inputs to C shift registers of segments 0 to L-1 


SRClock 


G 


A pulse on SRCIock[N] (ShiftRegisterClock N) loads the current 
values from ColorData lines into the L segments in segment 
group N. 


PTransfer 


1 


Parallel transfer of data from the shift registers to the internal 
NozzleEnable bits (one per nozzle) 


SenseSegSelect 


G 


A pulse on SenseSegSelect N ANDed with data on 

Colorl Data[n] selects the sense lines for segment n in segment 

group N 


Tsense 


1 


Temperature sense 


Vsense 


1 


Voltage sense 


Psense 


1 


Resistivity sense 


Wsense 


1 


Width sense 


Logic GND 


1 


Logic ground 


Logic PWR 


1 


Logic power 


V- 


Bus 

bars 


Actuator Ground 


V + 


Actuator Power 


TOTAL 


18+2G + CL 
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15 Memjet Printhead Interface 

The printhead interface (PHI) is the means by which the processor loads the Memjet print- 
head with the dots to be printed, and controls the actual dot printing process. The PHI con- 
tains: 



• a LineSyncGen unit (LSGU), which provides synchronization signals for multiple 
chips (allows side-by-side printing and front/back printing) as well as stepper motors. 

• a Memjet interface (MJI), which transfers data to the Memjet printhead, and controls 
the nozzle firing sequences during a print. 

• a line loader/format unit (LLFU) which loads the dots for a given print line into local 
buffer storage and formats them into the order required for the Memjet printhead. 

The units within the PHI are controlled by a number of registers that are programmed by 
the processor. In addition, the processor is responsible for setting up ^appropriate 
parameters in the DMA controller for the transfers from memory to the LLFU. This 

includes loading white (all (Ts) into annronriare rnlnrx Hnrino th» cmh ™a „c 

so that the page has clean edges. 

The PHI is capable of dealing with a variety of printhead lengths and formats. In terms of 
broad operating customizations, the PHI is parameterized as follows: 

Table 15. Basic Printing Parameters 



Name 


Description 


Range 


MaxColors 


No of Colors in printhead 


1-4 


SegmentsPerXfer 


No of segments written to per transfer Is equal to the 
number of segments in the largest segment group 


1-9 


SegmentGroups 


No of segment groups in printhead 


1-4 



The internal structure of the PHI allows for a maximum of 4 colors, 9 segments per trans- 
fer, and 4 transfers. Transferring 4 colors to 9 segments is 36 bits per transfer, and 4 trans- 
fers to 9 segments equates to a maximum printed line length of 18 inches. The total 
number of dots per line printed by an I 8-inch 4 color printhead is 1 15,200 ( 18 x 1600 x4). 

Other example settings are shown in Table 16: 
TableM6^Example Settings for Basic Printing Parameters 









rXfer 


ups 










e 

a 


mtsPe 


mtGro 


Bits per Xfer 




Printer Length 


Printer Type 


MaxCc 


E 

? 

CO 


Segme 


Comments 


4 inch CM Y 


Photo 


3 


8 


1 


24 




8 inch CMYK 


A4 /Letter 


4 


8 


2 


32 




X" inch C MY'h- 




0 
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15.1 Block Diagram of Printhead Interface 

The internal structure of the Printhead Interface is shown in Figure 47. 



LineSyncGen 
Unit 1 



LmeSyncI 



LS1 



Line 
Loader/Format 
Unit 



36 

-y- 



LmeSyncI Tc/Frorn 
-h> 1 — ^ other PHIs & 



stepper motor 





LineSyncGen 




To- Fron 


Unit 0 




DMA Controller 







- Master/Slave 



■P— 1-0« 



Line'ByncO 



^ To,- F rem 
\meSvncO otr-.e^PHls 



Memjet 
Interface 



Memjet 
Pnnthead 



Figure 47. Internal Structure of Printhead Interface 

In the PHI there are two LSGUs. The first LSGU produces LineSyncO, which is used to 
control the Memjet Interface in all synchronized chips. 'I he second LSGU produces 
LineSync 1 which is used to pulse the paper drive stepper motor. 

The Master/Slave pin on the chip allows multiple chips to be connected together for 
side-by -side printing, front/back printing etc. via a Master/Slave relationship. When the 
Master Slave pin is attached to V or > the chip is considered to be the Master, and LineSync 
pulses generated by the two LineSyncGen units are enabled onto the two tri-state Line- 
Sync common lines (LineSyncO and LineSync L shared by all the chips). When the Mas- 
ter Slave pin h attached to GND. the chip is considered to be the Slave, and LineSync 
pulses generated by the two LineSyncGen units are not enabled onto the common Line- 
Sync lines. In this way, the Master chip's LineSync pulses are used by all PHIs on all the 
connected chips. 

The following sections detail the LineSyncGen Unit, the Line Loader/Format Unit and 
Memjet Interface respectively. 



15.2 LineSyncGen Unit 

I he LineSyncGen units (LSGU) are responsible for generating the synchronization pulses 
required for printing a page. Each LSGU produces an external LineSync signal to enable 
line synchronisation. The generator inside the LGSU generates a LineSync pulse when 
told to *go\ and then every so many cycles until told to stop. The LineSync pulse defines 
the start of the next line. 

I he exact number of c> cles betw een LineSync pulses is determined by the CyciesBerween- 
Pulses register, one per generator. It must be at least long enough to allow one line to print 
( 1 00 (as or 200 us depending on whether the speed is low or high) and another line to load, 
but can be longer as desired (for example, to accommodate special requirements of paper 
transport circuitry ). If the CyclesBetweenPutses register is set to a number less than a line 
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print time, the page will not print properly since each LineSync pulse will arrive before the 



particular line has finished printing 



The follow ing interface registers are contained in the LSGU: 
Table 17 - LineSyncGen Unit Registers 



Register Name £ 


- i O XV ■ £ *jg^ ■ -■ * \v- * . ■ - - - - . ■■ „ - -„ v v — - - 

^fe^^^i^^^Description ■ f tp^^:.;:; 


CyclesBetweenPulses 


The number of cycles to wait between generating one Line- 
Sync pulse and the next. 


Go 


Controls whether the LSGU is currently generating LineSync 
pulses or not 

A write of 1 to this register generates a LineSync pulse, 
transfers CyclesBetweenPulses to CyclesRemaining. and 
starts the countdown. When CyclesRemaining hits 0, another 
LineSync pulse is generated, CyclesBetweenPulses is trans- 
ferred to CyclesRemaining and the countdown is started 
again. 

A write nf 0 tn thic ronict^r rt^n^ , _ _ 

.- _ ^ 31J ;o. oiupo me ^uumuuwii dtiu iiu irture 

LineSync pulses are generated 


CyctesRemaining 


A status register containing the number of cycles remaining 
until the next LineSync pulse is generated 



The LineSync pulse is not used directly from the LGSU. The LineSvnc pulse is enabled 
onto a tn-state LineSync line only if the Master/Slave pin is set to Master. Consequently 
the LineSync pulse is only used in the form as generated by the Master chip (pulses gener- 
ated by Slave chips are ignored). 



15.3 Memjet Interface 



The Memjet interface (MJl) transfers data to the Memjet printhead. and controls the noz- 
zle firing sequences during a print. 

The MJl is simply a State Machine (see Figure 48) which follows the printhead loadin- 
and firing order described in Section 14.2.I on paae 64. Section 14 2 2 on pa-e 6^ and 
includes the functionality of the Preheat Cycle and Cleaning Cvcle as described^ Section 
14.1.4 on page 61 and Section 14.1.5 on page 61. Both hiah-speed and low-speed printim- 
modes are available, although the MJl always fires a given nozzle from all segments in a 
printhead simultaneously (there is no separate firing of nozzles from one senment and then 
others). Dot counts for each color are also kept by the MJl. 



I he MJl loads data into the printhead from a choice of 2 do 



Ha sources: 



• All Is. I his means that all nozzles will fire during a subsequent Print cvcle. and is the 
standard mechanism for loading the printhead for a preheat or cleaning cycle. 

• f rom the 36-bit input held in the Transfer register of the l.LRf This is the standard 
means of printing an image. The 36-bit value from the LLI li is directK sent to the 
printhead and a I -bit - Advance" control pulse is sent to the LLFTJ. 

I he MJl knows how many lines it has to print tor the ra.'f \\>,.,, if... m n ; . .,i i , . . . 
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The MJI is therefore directly connected to the LLFU, LineSyncO (shared between all syn- 
chronized chips), and the external Memjet printhead. The basic structure is shown in Fig- 
ure 48. s 



[ LineSvrcO 





36 

^ ► 


▼ 


LLFU 


State 
Machine 





1 



Z 1 N/ Printhead ] 



Figure 48. Memjet Interface 

The MJI accepts 36 bits of data from the LLFU. Of these 36 bits, only the bits correspond- 
ing to the number of segments and number of colors will be valid. For example, if there 
are only 2 colors and 9 segments, bits 0-1 will be valid for segment 0, bits 2-3 will be 
invalid, bits 4-5 will be valid for segment 1, bits 6-7 will be invalid etc. The state machine 
does not care which bits are valid and which bits are not valid - it merely passes the bits 
out to the printhead. The data lines and control signals coming out of the MJI can be wired 
appropriately to the pinouts of the chip, using as few pins as required by the application 
range of the chip (see Section 15.3.1 on page 70 for more information). 



15.3.1 Connections to Printhead 

The MJI has a number of connections to the printhead. including a maximum of 4 colors, 
clocked in to a maximum of 9 segments per transfer to a maximum of 4 segment eroups. 
The lines coming from the MJI can be directly connected to pins on the chip', although not 
all lines will always be pins. For example, if the chip is specifically designed for onlv con- 
necting to 8 inch CMYK printers, only 32 bits of data need to be transferred each transfer 
pulse. Consequently 32 pins of data out (8 pins per color), and not 36 pins are required. In 
the same way. only 2 SRClock pulses are required, so only 2 pins instead of 4 pins are 
required to cater for the different SRClocks. And so on. 

It the chip must be completely generic, then all connections from the MJI must be con- 
nected to pins on the chip (and thence to the Memjet printhead) 

Table 18 lists the maximum connections from the MJI, man> of which are always con- 
nected to pins on the chip. Where the number of pins is variable, a footnote explains what 
the number of pins depends upon. The sense of input and output is w ith respect to the MJI. 
I he names correspond to the pin connections on the printhead. 

Table 18. Memjet Interface Connections 



Name 


#Pins 


I/O 


Description 


Chromapod Select 


3 


o 


Select which chromapod will fire (0-4) 


NozzieSelect 


4 


o 


Select which nozzle from the pod will fire (0-9) 


PodgroupEnable 


2 


o 


Enable the podgroups to fire (choice of 01, 10, 11) 


AEnable 


1 


o 


Firing pulse for podgroup A In the current design ail 
segments Tire simultaneously although multiple AEnable 
lines could be added for dividing the firing sequence 
over multiple segment groups for reasons of power and 
speed 
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Table 18. Memjet Interface Connections 



Ndmo 




On 


Description ^^^.R-S:-^:^:^^ 


BEnable 


1 


o 


Firing pulse for podgroup B In the current design all 
segments fire simultaneously, although multiple BEnable 
lines could be added for dividing the firing sequence 
over multiple segment groups for reasons of power and 
speed 


Color1Data[0-8] 


9 a 


o 


Output to Colon Data shift register of segments 0-8 


Color2Data[0-8] 


9° 


r\ 
w 


Output to Color2Data shift register of segments 0-8 


Color3Data[0-8] 


9 C 


o 


Output to Color3Data shift register of segments 0-8 


Color4Data[0-8] 


9 d 


o 


Oljtnilt to filter A. Ha ta chrft ron jctar r\ f r~ n <-«-. ^ ^ * r\ o 

uui h ui ^ ^unjitudid bum register or segments 0-8 


SRClock[1-4] 




4 e 


CJ 


A pulse on SRCIockfN] (ShiftRegisterClock) loads the 
current values from Colorl Data[0-8], Color2Data[0-8] 
Color3Data[0-8] and Color4Data[0-8] into the segment 
group N on the printhead 


PTra nsfer 


1 


o 


Parallel transfer of data from the shift registers to the 
orinthead's internal No77l^Fnahif> hit* /nno ^ r r^,,r«\ 


SenseSegSelect[1-4] 


4 f 


o 


A pulse on SenseSegSelect[N] AIMDed with data on 
Color1Data[n] enables the sense lines for segment n in 
segment group N of the printhead 


Tsense 


1 


I 


Temperature sense 


Vsense 


1 


I 


Voltage sense 


Rsense 


1 


I 


Resistivity sense 


Wsense 


1 


I 


Width sense 


TOTAL 


52 







15.3.2 



rl 1^?k y l T s , are mailable from the MJI, the number of pins coming from the ch,p will only 
ref ect the actual number of segments in a segment group. The pins for Colon Data are man 
datory, since each printhead must print in at least 1 color 

b Irc e A e ith« e «h o ,° nly translate , d ' nto f P fns if the chip is to control a printhead with at least 2 col- 
rni^9n^ 9h 5 " n f S ar ,f l v ?' lab!e from the WJI - the number of P' n s coming from the chip for 
Coior2Data will only reflect the actual number of segments in a segment group. 

C TheS A e u'I neS ? r S only translated in *° P^s if the chip is to control a printhead with at least 3 col- 
ors Although 9 lines are available from the MJI, the number of pins coming from the chip for 
Color3Data will only reflect the actual number of segments in a segment group 

d I^K Se 'u n S < are 0nly translated ,nt ° P ,n s if the chip is to control a printhead with 4 colors 
Although 9 lines are available from the MJI, the number of pins coming from the chip for 
Color4Data will only reflect the actual number of segments in a segment group 

6 ^flT^?K 4 lir l es l are ^ailable from the MJI, the number of pins coming from the chip will only 
reflect the actual number of segment groups A minimum of 1 pin is required since there is at 
least 1 segment group (the entire printhead) 

f Although 4 lines are available from the MJI, the number of pins coming from the chip will only 
reflect the actual number of segment groups A minimum of 1 pin is required Since there is at 
least 1 segment group (the entire printhead) 

Firing Pulse Duration 

The duration of firing pulses on the ALnable and BEnable lines depend on the viscosity of 
the ink (which is dependant on temperature and ink characteristics) and the amount of 
pouer available to the printhead. The t> pical pulse duration range is 1 .3 to I 8 u,s. The MJI 
therefore contains a programmable pulse duration table, indexed bv feedback from the 
printhead The table of pulse durations allows the use of a lower cost power supply, and 
akK in maintaining more accurate drop election 
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come from Tsense. Each entry is 8 bits, and represents a fixed point value in the range of 
0-4jas. The process of generating the AEnable and BEnable lines is shown in Figure 49. 



Vsense 



Tsense 



{> 



4 bit ADC 
(non-linear) 



4 -bit ADC 
(non-linear) 



4 



4 



Load Table 
/ 8 



Pulse 

Duration 

Table 
(256 x 8) 



8 



Pulse Width 
Generator 



AEnable 
BEnable 



Figure 49. Generation of AEnable and BEnable Pulse Widths 

The 256-byte table is written by the CPU before printing the first page. The table may be 
updated in between pages if desired. Each 8-bit pulse duration entry in the table combines: 

• User brightness settings (from the page description) 

• Viscosity curve of ink (from the QA Chip) 

♦ Rsense 

♦ Wsense 

* \ sense 

• Tsense 



15.3.3 Dot Counts 

T he VI J I maintains a count of the number of dots of each color fired from the printhead. 
The dot count for each color is a 32-bit value, individually cleared under processor con- 
trol. At 32-bits length, each dot count can hold a maximum coverage dot count of 17 
8-inch x ! 2-inch pages, although in typical usage, the dot count will be read and cleared 

after each page or half-page. 



The dot counts are used by the processor to update the QA chip in order to predict when 
the ink cartridge runs out of ink. The processor knows the volume of ink in the cartridge 
tor each of the colors from the QA chip. Counting the number of drops eliminates the need 
for ink sensors, and prevents the ink channels from running dry. An updated drop count is 
written to the QA chip after each page. A new page will not be printed unless there is 
enough ink left, and allows the user to change the ink without getting a dud half-printed 
page which must be reprinted. 
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The layout of the dot counter for Colorl is shown in Figure 50. The remaining 3 dot 
counters (Colorl DotCount, Coior2DotCount, and CoIor3DotCount) are identical in struc- 
ture. 



Colorl Data . 



9 






9 line to 4 

line 
encoder 


4 








-7^~ 






32 
,/ 



SRCiock[14j 



ClearCount 

^1 
^lear 



32 bit Coforl DotCount 
32 

— / 



32 

s 



Figure 50. Dot Count Logic 

15.3.4 Registers 

The processor communicates with the MJI via a register set. The registers allow the 
cessor to parameterize a print as well as receive feedback about printTprogress. 

The following registers are contained in the MJI: 



Table 19. Memjet Interface Registers 



Register Name 


Description 


Print Parameters 


SegmentsPerXfer 


The number of segments to write to each transfer. This also equals 
the number of cycles to wait between each transfer (before generating 
the next Advance pulse). Each transfer has MaxColors x Seg- 
mentsPerXfer valid bits 


SegmentGroups 


The number of segment groups in the printhead. This equals the num- 
ber of times that SegmentsPerXfer cycles must elapse before a single 
dot has been written to each segment of the printhead. The MJI does 
this 800 times to completely transfer all the data for the line to the 
printhead 


PrmtSpeed 


Whether to print at low or high speed (determines the value on the 
PodgroupEnable lines during the print) 


NumLines 


The number of Load/Print cycles to perform 


Monitoring the Print (read only from point of view of processor) 


Status 


The Memjet Interface's Status Register 


LinesRemammg 

Trans fers Remaining 


The number of lines remaining to be printed Only valid while Go=1 
Starting value is NumLines and counts down to 0 


The number of sets of SegmentGroups transfers remaining before the 
Printhead is considered loaded for the current line Starts at 800 and 
counts down to 0 Only valid while Go=1 


SegGroupsRemaming 


The number of segment groups remaining in the current set of trans- 
fers of 1 dot to each segment Starts at SegmentGroups and counts 
down to 0 Only valid while Go = 1 


SenseSegment 


The 9-bit value to place on the Colorl Data lines during a subsequent 
feedback SenseSegSelect pulse Only 1 of the 9 bits should be set, 
corresponding to one of the fmavmun-n 9 se^m^nts Sp^ e o,. , : . 
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Table 19. Memjet Interface Registers 



. RegfeterName 


T 1'^ : ^>^^.:rooscription - ; ' 


SetAIINozzies 


If non-zero, the 36-bit value written to the printhead during the Load- 
Dots process is all 1s, so that alt nozzles will be fired during the subse- 
quent PrintDots process This is used during the preheat and cleaning 
cycles. 

If 0, the 36-bit value written to the printhead comes from the LLFU 
This is the case during the actual printing of regular images 


Actions 


Reset 


A write to this register resets the MJI, stops any loading or printing 
processes, and loads all registers with 0 


SenseSelect 


A write to this register with any value clears the FeedbackValid bit of 
the Status register, and the remaining action depends on the values in 
the LoadingDots and PrintingDots status bits 

If either of the status bits are set, the Feedback bit is cleared and noth- 
ing more is done. 

If both status bits are clear, a pulse is given simultaneously on all 4 
SenseSegSelect lines with all ColorData bits 0 This stops any exist- 
ing feedback. Depending on the two low-order bits written to Sense- 
Select register, a pulse is given on SenseSegSe!ect1 , 
SenseSegSeiect2, SenseSegSelect3, or SenseSegSelect4 line, with 
the ColorlData bits set according to the SenseSegment register. 
Once the various sense lines have been tested, the values are placed 
in the Tsense, Vsense, Rsense. and Wsense registers, and the Feed- 
back bit of the Status register is set 


Go 


A write of 1 to this bit starts the LoadDots / PrtntDots cycles, which 
commences with a wait for the first LmeSync pulse. A total of Num- 
Lines lines are printed, each line being loaded/printed after the receipt 
of a LineSync pulse. The loading of each line consists of Segment- 
Groups 36-bit transfers As each line is printed. LinesRemaimng dec- 
rements, and TransfersRemaining is reloaded with SegmentGroups 
again. The status register contains print status information. Upon com- 
pletion of NumLines, the loading/printing process stops, the Go bit is 
cleared, and any further LineSync pulses are ignored During the final 
print cycle, nothing is loaded into the printhead 

A write of 0 to this bit stops the print process, but does not clear any 
other registers 


ClearCounts 


A write to this register clears the ColorlDotCount, Color2DotCount, 
Color3DotCount, and Color4DotCount registers if bits 0, 1 , 2, or 3 
respectively are set. Consequently a write of 0 has no effect 


Feedback 


Tsense 


Read only feedback of Tsense from the last SenseSegSelect pulse 
sent to segment SenseSegment is only valid if the FeedbackValid bit 
of the Status register is set 


Vsense 


Read only feedback of Vsense from the last SenseSegSelect pulse 
sent to segment SenseSegment Is only valid if the FeedbackValid bit 
of the Status register is set 


Rsense 


Read only feedback of Rsense from the last SenseSegSelect pulse 
sent to segment SenseSegment Is only valid if the FeedbackValid bit 
of the Status register is set 


Wsense 


Read only feedback of Wsense from the last SenseSegSelect pulse 
sent to segment SenseSegment Is only valid if the FeedbackValid bit 
of the Status register is set 


ColorlDotCount 


Read only 32-bit count of colorl dots sent to the printhead 


Color2DotCount 


Read only 32-bit count of color2 dots sent to the printhead 
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Table 19. Memjet Interface Registers 



> R^istwKariw4>>:t 


Read"only 32-bit count of co\o^ol& sent to the printfiead 


Color3DotCount 




Color4DotCount 


Read only 32-bit count of color4 dots sent to the printhead 



The MJPs Status Register is a l 6-bit register with bit interpretations as follows: 
Table 20. MJI Status Register 



Name 


Bits 


Description \ V ~ : 


LoadingDots 


1 


If set, the MJI is currently loading dots, with the number of dots 
remaining to be transferred in TransfersRemaining 
If clear, the MJI is not currently loading dots 


PrintingDots 


1 


If set, the MJI is currently printing dots. 

If clear, the MJI is not currently printing dots. 


PrintingA 


i 


» i..o un to aci wime inert; is a puise on tne Atnable line 


PnntingB 


1 


This bit is set while there is a pulse on the BEnable line 


FeedbackValid 


1 


This bit is set while the feedback values Tsense, Vsense, 
Rsense, and Wsense are valid. 


Reserved 


3 




PnntingChromapod 


4 


This holds the current chromapod being fired while the 
PrintingDots status bit is set. 


PnntingNozzles 


4 


This holds the current nozzle being fired while the 
PrintingDots status bit is set. 



The following pseudocode illustrates the logic required to load a printhead for a sin°le 
line. Note that loading commences only after the LineSync pulse arrives. This is to ensure 
the data for the line has been prepared by the LLFU and is valid for the first transfer to the 
printhead. 



for lir.eS v r. .: 
"rar.srersRer.a 



r.g = SCO to C 
r= ^ to Segrr.en tGroups 
i5e:Ai;t;o:;; e s) 

Set, all Colcr Data lines t c be 1 
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15.3.5 Preheat and Cleaning Cycles 

The Cleaning and Preheat cycles are simply accomplished by setting appropriate registers 
in the MJI: 



• SetAHNozzles = I 

• Set the PulseDuration register to either a low duration (in the case of the preheat mode) 
or to an appropriate drop ejection duration for cleaning mode. 

• Set NumLines to be the number of times the nozzles should be fired 

• Set the Go bit and then wait for the Go bit to be cleared when the print cycles have 
completed. 

The LSGU must also be programmed to send LineSync pulses at the correct frequency. 

15.4 Line Loader/Format Unit 

The line loader/format unit (LLFU) loads the dots for a given print line into local buffer 
storage and formats them into the order required for the Memjet printhead. It is responsi- 
ble for supplying the pre-calculated nozzleEnable bits to the Memjet interface for the 
eventual printing of the page. 

The printing uses a double buffering scheme for preparing and accessing the dot-bit infor- 
mation. While one line is being loaded into the first buffer, the pre-loaded line in the sec- 
ond buffer is being read in Memjet dot order. Once the entire line has been transferred 
from the second buffer to the printhead via the Memjet interface, the reading and writing 
processes swap butlers. The first buffer is now read and the second buffer is loaded up 
with the new line of data. This is repeated throughout the printing process, as can be seen 
in the conceptual overview of Figure 5 l . 



32 



Buffer 0 



Bufer 1 



Format 
Dots 



36 



To 
Memjet 
Interface 



3uffer0 



Q 



32 



3uffer 1 



Format 
Dots 



36 



To 
Memjet 
Interface 



Figure 51. Conceptual Overview of Double Buffering 
During Print Lines N and N + 1 

I he si/e of each buffer is !4KB\tes to cater for the maximum line length of 18 inches in 4 
colors (18 x 1600 x 4 bits - 1 15,200 bits - 14.400 bytes). The size for both Buffer 0 and 
Buffer 1 is 28.128 KBytes. While this design allows for a maximum print length of 18 
inches, it is trivial to reduce the buffer size to target a specific application. 
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The actual implementation of the LLFU is shown in Figure 52. Since one buffer is being 
read from while the other is being written to, two sets of address lines must be used. The 
->2-b,ts Dataln from the common data bus are loaded depending on the WriteEnabies 
which are generated by the State Machine in response to the DMA Acknowledges. 



To DMA Controller 



^'32 
Dataln 



Requests 



Buffer 0 



WnteEnables 
4 



Acknowledges 



13 



Address 
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✓ '4 
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9 entry x 4-bit shift register 



36 
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~* DataBus 
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MJf 



^ 36-bit Transfer register 



36 To 
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Figure 52. Structure of LLFU 

A multiplexor chooses between the two 4-bit outputs of Buffer 0 and Buffer 1, and sends 
the result to a 9-entry by 4-bit shift register. After a maximum of 9 read cycles (the number 
depends on the number of segments written to per transfer), and whenever an Advance 
pulse comes from the MJI, the current 36-bit value from the shift register is gated into the 
j>6-bit Transfer register, where it can be used by the MJI. 

Note that not all the 36 bits are necessarily valid. The number of valid bits of 36 depends 
on the number of colors in the printhead, the number of segments, and the breakup of seg- 
ment groups (if more than one segment group). For more information, see Section 14 2 on 
page 63. 

A single line in an A-inch C-coior pnnthead consists of 16001. C-color dots. At I bit per 
colored dot, a single print-line consists ol 1600LC bits The I LI L' ,s capable of address- 
ing a maximum line si/e of 18 inches m 4 colors, which equates to 108,800 bits (14 
KBwes) per line These bits must be supplied to the MJI in the correct order for bein- sent 
on to the pnnthead. See Section 14.2.1 on page 64 for more information concerning the 
Load Cycle dot loading order, but in summary. 2LC bits are transferred to the printhead in 
SexmentGroups transfers, with a maximum of 36 bits per transfer Each transfer to a par- 
ticular segment of the printhead must load all colors simultaneously. 
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15.4.1 Buffers 

Each of the two buffers is broken into 4 sub-buffers, I per color. The size of each 
sub-buffer is 3600 bytes, enough to hold 18-inches of single color dots at 1 600 dpi. The 
memory is accessed 32-bits at a time, so there are 900 addresses for each buffer (requiring 
10 bits of address). 



All the even dots are placed before the odd dots in each color's buffer, as shown in Figure 
53. If there is any unused space it is placed at the end of each color's buffer. 



Buffer 0/1 



Colorl Dots Buffer 



Color2 Dots Buffer 



Color3 Dots Buffer 



Color4 Dots Buffer 



Even Dots 



Odd Dots - 



Figure 53. Conceptual Structure of Buffer 

1 he amount of memory* actually used is directly related to the printhead length. If the print- 
head is 18 inches, there are 1800 bytes of even dots followed by 1800 bytes of odd dots, 
with no unused space. If the printhead is 12 inches, there are 1200 bytes of even dots fol- 
lowed by 1200 odd dots, and 1200 bytes unused. 

1 he number of sub-buffers gainfully used is directly related to the number of colors in the 
printhead This number is typically 3 or 4, although it is quite feasible for this system to be 
used in a 1 or 2 color system (with some small memory wastage). In a desktop printing 
environment, the number of colors would be 4: Colorl -Cyan, Color2 = Magenta, 
Color3"- Yellow, Color4 = Black. 

The addressing decoding circuitry is such that in a given cycle, a single 32-bit access can 
be made to all 4 sub-buffers - either a read from all 4 or a write to one of the 4. Only one 

bit of the 32-bits read from each color buffer is selected, for a total of 4 output bits. The 
process is shown in Figure 54. 15 bits of address allow the reading of a particular bit b> 
means of 1 0-bits of address being used to select 32 bits, and 5-bits of address choose 1 -bit 
trom those 32. Since all color buffers share this logic, a single 15-bit address gives a total 
of 4 bits out, one per color. Hach buffer has its own WriteEZnable line, to allow a single 



( ; onf identic! 



7 September 1 999 



78 



Stlverbrook Research 



Netpage Printer Design Description 



draft vO 3 



32-bit value to be written to a particular color buffer in a given cycle. The 32-bits of 
Dataln are shared, since only one buffer will actually clock the data in. 
Address 15 c 



Write Color! 



10 



Dataln 32 




Write Color2 



Write Color3 



Write Cclor4 



Coiorl Dots Buffer 



Color? Dots Buffer 



Color3 Dots Buffer 



Color4 Dots Buffer 



32 



32 



1 



32 
► 



32 
► 



4 

► 



Figure 54. Logical Structure of Buffer 

Note that regardless of the number of colors in the printhead, 4 bits are produced in a 
given read cycle (one bit from each colons buffer). 



15.4.2 Address Generation 
15 A. 2.1 Reading 



Address Generation for reading is straightforward. Each cvcle we generate a bit address 
which is used to fetch 4 bits representing l-bit per color for a particular segment By add- 
ing 400 to the current bit address, we advance to the next segment's equivalent dot We 
add 400 (not 800) since the odd and even dots are separated in the buffer We do this firstly 
SegmentGroups sets of SegmentsPerXfer times to retrieve the data represents the even 
dots (the dot data is transferred to the MJI 36 bits at a time) and another SegmentGroups 
sets of SegmentsPerXfer times to load the odd dots. This entire process is repeated 400 
times, incrementing the start address each time. Thus all dot values are transferred ,n the 
order required b> the pnnthead in 400 x 2 x SegmentGroups x SegmentsPerXfer cycles. 

In addition, we generate the TransferWriteEnable control signal. Since the LI FU starts 
before the MJI, we must transfer the first value before the Advance pulse from the MJI 
We must also generate the next value in readiness for the first Advance pulse The solution 
is to transfer the first value to the Transfer register after SegmentsPerXfer cycles and then 
to stall SegmentsPerXfer-cycles later, waiting for the Advance pulse to start the next Seg- 
mentsPerXfer cvcle v-roup Once the first Advance pulse arrives the f f n * ;■ - • < 



I iie read proce^ i> sinus n in the following pseudocode 
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DoneFirst = FALSE 

For Dot InSegment 0 = 0 to 400 

CurrAdr = Dot InSegmentO 

XfersRenaming = 2 x Segment Groups 

Dot Co ant = SegmentsPerXfer 

Do 

VI = DotCcunt - 0 

Trans ferWr iteEnable = {VI AND NOT DoneFirst) OR ADVANCE 
Stall = VI AND (NOT Trans f erWr i teEnable ) 
If (NOT Stall) 

Shift Fegister-Fetch 4-bits f r cm Cur rRead3u f f e r : Cu r r Adr 
CurrAdr - CurrAdr + 400 
If (Vi) 

DotCour.t = S e gne ntsPerXter - 1 
> fe r s Regaining = X f e r s Rerria l r.i ng - 1 
Else 

DotCount = DotCount - 1 
End If 
End I f 

Until ( X t ersPemaining=0 ) AND (NOT Stall) 
End For 



The final transfer may not be fully utilized. This occurs when the number of segments per 
transfer does not divide evenly into the actual number of segments in the printhead. An 
example of this is the 8'/ 2 " printhead, which has 17 segments. Transferring 9 segments 
each time means that only 8 of the last 9 segments will be valid. Nonetheless, the timine 
requires the entire 9th segment value to be generated (even though it is not used). The 
actual address is therefore a don't care state since the data is not used. 

Once the line has finished, the CurrReadBuffer value must be toggled by the processor. 

15.4.2.2 Writing 

The write process is also straightforward. 4 DMA request lines are output to the DMA 
controller. As requests are satisfied by the return DMA Acknowledge lines, the appropri- 
ate 8-bit destination address is selected (the lower 5 bits of the 15-bit output address are 
don i care values) and the acknowledge signal is passed to the correct buffer's WriteEnable 
control line (the Current Write Buffer is -CurrentReadBuffer). The 10-bit destination 
address is selected from the 4 current addresses, one address per color. As DMA requests 
are satisfied the appropriate destination address is incremented, and the corresponding 
TransfersRemaining counter is decremented. The DMA request line is only set when the 
number of transfers remaining for that color is non-zero. 

The following pseudocode illustrates the Write process: 



"ur rent Adr : : - 4 j = 0 

Wr.ile ( Colo r> f er 5?.ena : n: nq [ 1 - 4 ; are non-r_ero) 

DMA request [ 1-4 ] - Co 1 o r X f e r s Rema i n i ng [ 1 - 4 ] NOT - 0 
If DMAA knowledge [ N ; 

CurrWr i te3 fer -Current Adr {N] -- Fetch 3 2 -bits f : or, • : i * i bos 

CcicrXfers 3 err. a : n i ng ; N ; - Col orXfersRerr.a :r.:r.g[ N ] - 1 (floor 0 > 



C:r ; (jemu 
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15.4.3 Registers 

The following interface registers are contained in the LLFU: 



v ; Register Name ^ly ^ 




SegmentsPerXfer 


iiuinuet ox segments wnose dots must De loaded before 
each transfer. This has a maximum value of 9. 


SegmentGroups 


The number of segment groups in the printhead. This has a 
maximum number of 4. 


CurrentReadBuffer 


The current buffer being read from. When BufferO is being 
read from, Bufferl is written to and vice versa. 

Should be toggled with each AdvanceLine pulse from the 
MJI. 


Go 


Bits 0 and 1 control the starting of the read and write pro- 
cesses respectively. 

A non-zero write to the appropriate bit starts the process. 


Stop 


Bits 0 and 1 control the stopping of the read and write pro- 
cesses respectively. 

A non-zero write to the appropriate bit stops the process. 


Stall 


This read-only status bit comes from the LLFU's Stall flag 
The Stall bit is valid when the write Go bit is set. 




A Stall value of 1 means that the LLFU is waiting for the 
ADVANCE pulse from the MJI to continue. The CPU can 
safely start the LSGU for the first line once the Stall bit is set 


ColorXfersRemaming[1 -4] 


The number of 32-bit transfers remaining to be read into the 
specific ColorfN] buffer 



15.5 Controlling a Print 



When controlling a print the CPU programs and starts the LLFU in read mode to ensure 
that the first line of the page is transferred to the buffer. When the interrupts arrive from 
the DMA controller, the CPU can switch LLFU buffers, and program the MJI. The CPU 
then starts the LLFU in read/write mode and starts the MJI. The CPU should then wait a 
sufficient period of time to ensure that other connected printer controllers have also started 
their LLFUs and MJIs (if there are no other connected printer controllers, the CPU must 
wait until the Stall bit of the LLFU is set, a duration of 2SegmentsPerXfer cycles). The CPU 
can then program the LGSU to start the synchronized print. As interrupts arrive from the 
DMA controllers, the CPU can reprogram the DMA channels, swap LLFU buffers, and 
restart the LLFU in read write mode. Once the LLFU has effectively filled its pipeline, it 
\mII stall until the next Advance pulse from the MJI The MJI does not have to be touched 
during the print 



H tor some reason the CPU wants to make any changes to the MJI or LLFU registers dur- 
ing an inter-line period it should ensure that the current line has finished printing loadine 
by polling the status bits of the MJI and the Go bits of the LLFU 
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S.lve r brook Research Netpa ge Business Implementation Procedures and Equipment 

1 Overview 

This document describes business implementation procedures and equipment which lever- 
age the Netpage System [ I ]. 

1.1 Notation 

In the following sections, diagrams are used to illustrate the interactions between the vari- 
ous participant roles in a business procedure. Fach participant is shown as a pair of over- 
lapping rectangles. The front rectangle is labelled with the name of the participant The 
back rectangle ,s used as the source and/or target of actions. Hach action is shown as an 
arrow joining a subject participant with an object participant. Fach action arrow is labelled 
with a description of the action. 

Actions are spatially arranged so that time proceeds from top to bottom and left to ri»ht 
Later actions may coincide with, but never be earlier than, "earlier" actions. 

Any action is understood to be "caused- by the immediately prior action which the subject 
ot the action was an object of. In the absence of a prior action, the action is understood to 

be spontaneous. 
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2.1 



Netpage Publisher 

Advertising Fees 



A publisher who publishes to on-demand printers can receive advertising fees on advertis- 
ing placed within their publications. 

This can provide a profitable role for a publisher who publishes to on-demand printers, 
and can allow the publisher to attract readers by being able to provide them with subsi- 
dized or free publications. 

A publication can be automatically delivered to a subscribing user's printer on a regular 
basis by the publisher. The publication can also be delivered to a user's printer ad hoc at 
the user's request. The publication can be printed as interactive paper 

As illustrated in f igure I, an advertiser places an advertisement with a publisher; the pub- 
lisher eventually prints the advertisement on a user's on-demand printer, typically as part 
of one of the publisher's publications; the publisher charges the advertiser an agreed 
advertising fee; the advertiser pays the publisher the advertising fee. 

I or practical purposes, advertising fees can be negotiated, invoiced and settled in bulk. 











^ 



on-demand 

prlitVf- 



Figure 1. Payment of Advertising Fee 

2.2 Click-Through Fees 

A publisher who publishes on interactive paper can receive click-through fees on advertis- 
ing placed within its publications. 

I his can provide a profitable role for a publisher who publishes on interactive paper, and 
can allow the publisher to attract readers by being able to provide them with subsidized or 
tree publications. 

As illustrated in f igure 2. an advertiser places a hvperlmk with a publisher, the publisher 
eventually prints the hyperlink, typically as part of one of the publisher's publications; 
when a user clicks on the hyperlink, for example to request a linked document from the 
advertiser, the advertiser is notified, the advertiser pays the publisher an agreed 
click -through fee 

for practical purposes, click-through fees can be negotiated, invoiced and settled in bulk, 
and mav pari of \s ider advertising agreements 
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2.3 Sales Commissions 



A publisher who publishes on interactive paper can receive commissions on sales initiated 
through advertising placed within its publications. 

This can provide a profitable role for a publisher who publishes on interactive paper and 
can allow the publisher to attract readers by being able to provide them with subsidized or 
free publications. 

As illustrated in Figure 3, an advertiser places a hyperlink with a publisher; the publisher 
eventually prints the hyperlink, typically as part of one of the publisher's publications- 
when a user clicks on the hyperlink to request a linked document from the advertiser the 
advertiser is notified; when the user eventually makes an online purchase via the linked 
document (or via a document obtained via the linked document), the advertiser is notified; 
the advertiser pays the publisher an agreed commission on the sale. 

The sales commission may be a fixed fee or may be based on the value of the sale. 

1 or practical purposes, sales commissions can be negotiated, invoiced and settled in bulk, 
and may part of wider advertising aiireements. 
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Figure 3. Payment of Sales Commission 
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3 Netpage Printer Provider 

3.1 Commissions on Advertising Fees 

A provider of an interactive printer can receive a percentage of advertising fees earned 
directly through the printer. 

This can provide a profitable role for a provider of interactive printers, and can allow the 
provider to attract users by being able to partially or fully subsidize the capital and running 
costs of each user's printer. 

As illustrated in Figure 4, a printer provider provides a user with a printer; an advertiser 
places an advertisement with a publisher; the publisher eventually prints the advertisement 
on the printer, typically as part of one of the publisher's publications; the publisher 
charges the advertiser an agreed advertising fee; the advertiser pays the publisher the 
advertising fee; the publisher pays the printer provider an agreed commission on the 
advertising fee. 



provide phnte-r 




Figure 4. Payment of Commission on Advertising Fee 

3.2 Commissions on Click-Through Fees 

A provider of an interactive printer can receive a percentage of click-through fees earned 
direct I v through the printer. 

I his can provide a profitable role for a provider of interactive printers, and can allow the 
provider to attract users by being able to partially or fully subsidize the capital and running 
costs of each user's printer. 

A - M'i: " !••• j jn l •> , p r -in?,.r l.ti „ r - , ,. ... . , ;.i , ,. ; ... 
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P tn ^ ider an agreed commission on the c I ick -through lee 
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Figure 5. Payment of Commission on Click-through Fee 



3.3 Commissions on Sales Commissions 

A provider of un interactive printer can receive a percentage of sales commissions earned 
directly through the printer. 

(his can provide a profitable role for a provider of interacts e printers, and can allow the 
provider to attract users by being able to partially or fully subsidize the capital and running 
costs of each user's printer 

As illustrated in figure 6. a printer provider provides a user with a printer; an advertiser 
places a hvperlink with a publisher; the publisher eventualK prints the hyperlink. typicallv 
as pan of one of the publisher's publications; when a user clicks on the hyperlink to 
request a linked document from the advertiser, the advertiser is notified; when the user 
eventually makes an online purchase via the linked document (or via a document obtained 
via the linked document), the advertiser is notified; the advertiser pays the publisher an 
agreed commission on the sale; the publisher pays the printer provider an agreed commis- 
sion on the sales commission. 



2/ October 1999 



7 




Sftverbrook Research Netpage Business Implementation Procedures and Equipment draftvO 



printer 
provider 



provide printer- 



place hyperlink 



print 
hyperlink 



page 



click on 
hyperlink 



notify merchant 



print 
hyperlink 



printer 



publisher 




notify merchant 



pay sale;; rommi^jion 



pay sales commission commission 
Figure 6. Payment of Commission on Sales Commission 
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1 Pen Controller Architecture 

The Netpage Pen controller consists of a controlling processor, a radio transceiver, a tilt 
sensor, a nib pressure sensor, a IR image sensor, flash memory, and 512KB of DRAM, as 
illustrated in F : igure 1 . 




processor address, data and control bus 




Figure 1. Pen controller architecture 

The controlling processor captures and decodes IR position tags from the page via the 
image sensor, monitors the tilt and pressure sensors, controls the autofocus voice-coil, 
controls the user interface (tricolor LFiD). and handles wireless communication with the 
local Netpage Printer. It consists of a medium-performance (~~25MHz RISC) general-pur- 
pose microprocessor. 

Tuo-a\is tilt sensing is provided by a two-axis accelerometer. Nib pressure sensing is pro- 
vided by an IR emitter and photodiode pair in conjunction with a reflector coupled with 
the sprung nib. 

The processor, digital transceiver components, 96x96 image sensor, flash memory and 
5 12KB DRAM are integrated in a single ASIC. The MLMS and analog RF components, 
accelerometers, and the IR emitter photodiode are integrated in a single Ml: MS chip, also 
used in the Netpage Printer. 

The radio transceiver communicates in the unlicensed 900MHz band normally used by 
cordless telephones, and uses frequency hopping and collision detection to provide inter- 
t e re n c e - 1 r e e c o m nui n i c a t i o n . 
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2 Netpage Encoding 

Hidden information on a printed Netpage encodes the unique id of the page instance, 
together with position information which tells the pen which part of the page the user is 
interacting w ith. The Netpage Network can correlate any reference to any part of any page 
ever printed with the contents of the page, solely on the basis of the page id and the posi- 
tion. 

The entire page is tiled with position tags. Each position tag must be independent, to allow 
for single-point virtual button and hyperlink presses, and so contains both the local x-y 
position and the global page id. In addition particular flags are associated with each tag, 
such as whether the pen should turn on its light while over the tag. 

The tag data (page id, a local x-y position, and control flags) is error-correctably encoded 
as a square binary matrix which is printed as an array of bi-level macrodots. Each macro- 
dot itself consists of a square of one or more printed dots to ease image processing. 

The position encoding supports a particular page size. Where the physical page size 
exceeds this virtual page size, the physical page is simply tiled with multiple virtual pages, 
each w ith a different page id. Figure 2 illustrates the concept of a page tiled w ith position 
tags. The contents of each tag are not shown on this illustration. 




Single Tag 



Figure 2. Part of a page tiled with position tags 

The tags are printed with an infrared-absorptive ink that can be read by the pen device. 
Since black ink is IR absorptive, limited functionality can be provided on offset-printed 
pages using black ink on otherwise blank areas of the page - for example to encode but- 
tons. Alternativelv an invisible infrared ink can be used to print the position tags over the 
top of a regular page. However, if invisible IK ink is used, care must be taken to ensure 
that an) other printed information on the page is printed in int'mrcd-ir anspcireni CMY ink, 
for black ink will obscure the infrared tags. T he monochromatic scheme was chosen to 
maximize dynamic range in blurry reading env ironments. 

Simple virtual button presses don't specificalk require invisible ink. Buttons could be vis- 
ibly coded since the real estate is available. Hyperlinks require invisible ink, but impose 

no other requirements on the tag encoding 
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Table 1. Encoding parameters 



parameter 


1 --fH<t.; : ^ 


Jv-^ - Y « : 


P 


virtual page width (inches) 


12 a 


d 


dot pitch (dots per inch) 


1600 


k 


macrodot width (dots) 


4 


b 


registration border width (macrodots) 


2 


r 


data redundancy factor 


15 


m 


coordinate precision (bits) 




9 


page id precision (bits) 




s 


tag width (macrodots) 




n 


tags per page b 





a. US Letter page length 
b per dimension 

Given the encoding parameters defined in Table 1, the tag width is given by: 
5 = 6+- Jr(2m + g) 

Given a particular tag width, the number of tags per page in each dimension is given by: 

RA 

ks 

This in turn y ields the required coordinate precision: 



If 10 billion people all generate 100,000 pages per annum for 1000 years, they will gener- 
ate 10 ^ pages, or approximately 2 60 pages. A page id precision of 64 bits should therefore 
be sufficient, notwithstanding issues of efficient contiguous page id allocation. 

Assuming a page id precision of 64 bits, and other parameter values as given in Table 1, 

the equations converge on a tag width of I 7 macrodots, 283 tags per page dimension, and 
a coordinate precision of 9 bits. This precision supports a maximum pa-e dimension of 
2 1 .7" or 552mm 

Given a particular tag width, the position resolution, in mill 'im^wrs . is m\en by: 

25.4*5 
d 

and, in points, is given by: 

72A.Y 
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Magazine-qualit> printed text normally has a size of 10 points. Forms tilled in by hand 
normally allow for handwritten text with a size of about 20 points. A position resolution of 
3 points therefore translates to a character-oriented resolution of 3.3 for printed text and 
6.6 for handwriting recognition. This position resolution is increased to the 200 dpi 
required for handwriting recognition by taking into account the position of the tag within 
the captured image area. 

With a position tag width of 17 macrodots, and 4x4 dots (at 1600 dpi) per macrodot, the 
raw position tag gives about 24 positions per inch. Consequently any positioning scheme 
must achieve 10 times better resolution based on tag position within the sensed image. 



Position Tag Structure 

Each position tag encodes the following information: 



Table 2. Data Encoded in a Position Tag 



Name 


#Bits 


Description 


Pageld 


64 


Defines the Netpage page instance id 


X 


9 


Defines the x coordinate within the given Pageld 


y 


9 


Defines the y coordinate within the given Pageld 


ActiveArea 


1 


Defines whether the pen should turn its light on while over 
the tag 


PressureSensitiveArea 


1 


Determines whether the pen should return continuous 
pressure or not while over the tag 

0 = don't return pressure readings 

1 = return pressure readings 


Reserved 


6 


For future use Store as 0 for this version 


TOTAL 


90 





Fach position tag is a 17x17 array of macrodots, with each macrodot being a 4x4 square 
of 1600 dpi dots, as shown Figure 3. 




1 7 macrodots 



Figure 3. Macrodots in a Position Tag 

Since the dots are monochrome I R-absorptive dots, printed on an I R -reflective back- 
ground, a "black macrodot" is physically different from a "unite macrodot". Figure 4 
illustrates a magnified view of macrodots 
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Black and 
white macrodots 



Black macrodot 
surrounded by white 

Figure 4. Magnified Macrodots 




White macrodot 
surrounded by black 



The position tag structure is based on concepts used in the public-domain Aztec 2D Bar 
code [7]. invented by Andy Longacre of Welch Allyn Inc. in 1995. While the position ta« 
uses the same bull's-eye structure mechanism from the Small Aztec Symbol for symbol 
location and orientation, we define our own interpretation for the mode bits and do not 
allow variable length structures. In addition, the position tag is a 17x17 array, which is not 
possible in a pure Aztec code. 

Figure 5 shows the high-level structure of a position tag in relation to its 17x17 macrodot 
array. The target/orientation area consists of a 5-layer bull's-eye with 3-bit orientation 
markers on the outer corners. The format of this area is identical to that used in the Small 
Aztec Barcode although it is placed off-center. The Mode Data Area is the area between 
the orientation markers in what would otherwise be the 6th layer of the bull's-eye tara et 
The remaining area is used to hold the actual data for the position tag 



12 



Legend 

n Target / orientation 
■ area (fixed pattern) 

E3 Mode Data Area 
Data Area 



Orientation Markers 
Top Left 



Top Right 
Lower Left 
Lower Right 



Figure 5. High Level Structure of a Position Tag 



2.1.1 Target and Orientation Area 



I he target and orientation area consists of the fixed pattern shown in black and unite in 
Figure .v I he format ot this area is identical to that used in the Small Aztec Barcode 

The black posmons in are black macrodots. and the white positions are unite macrodots 
he 5-layer bull's-eye pattern is easily found in a 2D image bv scans for topolosical con- 
nectivity, and ,s then useful for pinpointing the exact center regardless of orientation and 
tor determining the main axes and local x-v dimensions. On the four comers of the target 
are 3-b,t clusters of orientation bits, which allow the tag's orientation to be quicklv deter- 
mined. 
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2.1.2 



Mode Data Area 



The bits in the layer immediately adjoining the target (other than the orientation bits) com- 
prise a 28-bit string, starting upper left and circling clockwise. This area is known as the 
Mode Data Area. The relative position of the 28 Mode Data bits can be seen in Figure 6. 



Mode Data bit 0 
Mode Data bit 27 




Target/Orientation Area 



Figure 6. Mode Data Area 

The 28-bit string encodes meta-information used to decode the remainder of the position 
tag. Of particular note is the first bit of the meta information. If this bit is 0, then the 
decoding of the data area is as described here. Setting this bit to I allows for redefinition 
of the position tags at some later date while remaining compatible with existing pages. 
The meta information and its encoding to 28 bits is described in Section 2.2. 1 . 



2.1.3 



Data Area 



When the first bit of the meta information decoded from the Mode Data Area is 0, then the 
data area is defined to be the I 68 bits as shown in F : i^ure 7. 



The encoded message data is placed onto the tag in 2-ceIl high layers, starting at the upper 
lett and spiraling clockwise out to the edge of the symbol. The data is therefore interpreted 
as a sequence of related ' k dominos , \ each I w ide and 2 tall, with their more significant bit 
always further from the target area (a black macrodot represents a "T 1 bit, and a white 
macrodot represents a "0" bit). Figure 7 illustrates the sequence and orientation of the 
dominos when turning the corners in the data area. 



Figure 7. Domino Mapping and Sequencing of Data Area 

I he s>stematie domino-based layout of the data simplifies both encoding and decoding at 
the graphical level. The encoding mechanism for the data is described in Section 2.2.2, 
although it is worth noting here that decoding errors and tag damage are expected to occur 
more towards the edges of the tag. 
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Note that the 17x17 structure of the position tag means that with reference to the strict 
Aztec Code, the first layer of data is complete, but the second layer of data is only half 
full. If the second layer was completely full the position tag would be a 19x19 square cell 
with the target area in the center. 

2.2 Position Tag Encoding 

2.2.1 The Mode Message 

The 28-bit string in the Mode Area is used to hold 8 mode bits. These 8 mode bits describe 
how the remainder of the position tag bits are encoded. 

The 8 mode bits are encoded into 28 bits via Reed-Solomon encoding. The Mode Area 
therefore holds the 8 mode bits and 20 additional check bits. The 8 mode bits are parsed 
into two 4-bit words, and then 5 additional check words are computed by systematic 
Reed-Solomon encoding over the Galois field GF(l6) based on a prime modulus polyno- 
mial of: x 4 + x + l (=19 decimal). The generator polynomial of (.v-2 1 )..(x-2 4 ) is: 

4 . . 3 ,2 
X + i 1 X + 4X +- OA + I 

Reed-Solomon encoding is chosen for its ability to deal with burst errors and effectively 
detect and correct errors using a minimum of redundancy. Reed-Solomon coding is dis- 
cussed in detail in [8], [4], and [3]. The reader is advised to refer to these sources for back- 
ground information. 

Table 3 shows the interpretation of the mode bits. Note that the first bit of the mode data is 
a version bit, and determines the interpretation of the remainder of the mode bits and the 
tag data. The Version bit should therefore be set to 0. Later interpretations of the tag can be 

defined at a later date via the version bit. 



Table 3. Interpretation of Mode Bits 



#Bits 


Name 


Description 




Version 


0 = the interpretation of the remaining 7 bits and the data area is 
as described in this document 

1 = reserved for future use 


1-2 


NumMsgWords 


Defines how many 6-bit codewords of the data area are message 
codewords, and how many are check-word codewords. This num- 
ber is determined by the encoding scheme used for data (see 
Section 2 2 2) Values are 

00 - 15 data message codewords 

01 = 16 data message codewords 
10 = 17 data message codewords 
11=18 data message codewords 


3-7 


Reserved 


0 



2.2.2 The Data Message 

The Data Area contains 168 raw bits, which is an encoded form of the 90 bits of position 
tau data (see Table 2) The I 68 bits are represented by I 04 bits in the first layer surround- 
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The second step Reed-Solomon encodes the data over the Galois field GF(64), which gen- 
erates additional 6-bit check words. 

2.2.2.1 Step 1: Message Encoding 

In the first step of message encoding, the 90 data bits are arranged into a sequence of 6-bit 
message codewords in a generally direct fashion, starting at the most significant bit of the 
first codeword, with two key exceptions: 

• whenever the first 5 bits placed in a codeword are all "0"s, then a dummy "l" is 
inserted into that codeword's LSB and the following message bit starts off the next 
codeword. 

• whenever the first 5 bits placed in a codeword are all "Ps, then a dummy k 'CT is 
inserted into that codeword's LSB and the following message bit starts off the next 
codeword. 

In the end, the character and byte boundaries in the original 90-bit message have no neces- 
sary relationship with the codeword boundaries. Up to 5 bits may remain unfilled in the 
final message codeword, and they are padded out with " Ps (and possibly a final dummy 
"O^ if necessary ) to eliminate any ambiguity. 

The code-forming rules are designed to never create a message codeword of all 0s or all 
I s, but the error encoding adds on additional codewords of any value. Thus during decod- 
ing, an occurrence of those illegal values w ithin the message region (but not within the 
check region) can be regarded as a correctable erasure (this is more useful than not know- 
ing which codeword is in error). 

In the case of our 90-bit original messages, there are two cases to consider: 

• the best case is that 15 codewords are generated for the data message 

• the worst case is that 18 codewords are generated for the data message 

The codeword based message therefore consumes D 6-bit codewords, where D is in the 
range 15-18 

2 2.2.2 Step 2: Generate Check Words 

The second step of message encoding involves generating the check words for the data 
message. Since the data message consumes between 15 and 18 6-bit codewords of the 28 
available codewords, the remaining 1 3 to 10 codewords can be used for check words. 

The K additional check words are computed by systematic Reed-Solomon encoding over 
the Galois field GF(64) based on a prime modulus polynomial of: i - + v + 1 ( 67 deci- 
mal). The generator polynomial is simply expanded out to {x-2 l )..(x-2 K ) as needed at the 
time of printing. The check words are not adjusted in the same way as the message bits are 

adj usted. 

2.2.2.3 Step 3: Place into Data Area 

The final message to be stored in the data area of a position tag consists of 28 6-bit code- 
words made up of D message codewords followed by K check codewords. The exact num- 
ber of message codewords, D, is stored into the 2 -bit Num Msg Words component of the 
Mode information for the position tag. 
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The resulting sequence of codewords is parsed anew into a sequence of 2-bit dominos (see 
Figure 8) and is then taken in reverse order and graphically arranged spiralling clockwise 
and outwards through the data area (see Figure 7). As decoding errors and tag damage are 
expected to occur more towards the edges of the tag, by reversing the codeword sequence, 
the message codewords occupy those edge regions. 



MSB 



LSB 



6 -bit codeword 



5 


4 


3 


2 


1 


0 



4 


2 


0 


5 


3 


1 



3 2-bit dominos 



Figure 8. Relationship of 6-bit codeword to generated dominos 

One macrodot is placed into the data area per bit. The placement of an IR-absorptive ink 
macrodot represents a " T\ If there is no IR-absorptive ink placed on the page at the partic- 
ular location, then that macrodot represents a "0". 
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3 In-Pen Processing 

The pen must send the printer 100 samples per second. The mechanism for reading posi- 
tional information from the position tags must work at this 100 Hz rate. The following 
tasks must be performed each sample: 

• capture time, tilt and pressure 

• capture the image 

• locate the position tag 

• decode the position tag 

• build stroke information 

• activate the light as required 

• encry pt the stroke data 

• transmit the encrypted data 

When a pen-down is registered, we capture the first three frames at the tw o extreme focus 
values and the middle focus value. After that we perform regular auto-focus based on a 
sharpness metric. In this way we are more likely to capture a "single-click" event. 

While the pen is in range of the printer, partially completed strokes are transmitted as they 
are being formed. When the pen moves out of range, stroke information is buffered within 
the pen (approximately 12 minutes of pen motion while physical ly on the page), and trans- 
mitted later. 

Elach pen contains a characteristics block which can be read by the printer. It contains at 
least the information shown in Table 4. 



Table 4. Pen characteristics block 



Name 


* v Description 


Penld 


A unique pen identifier - unique across all pens 




Modelld 


The model number of this pen 3 




Manufactured 


The manufacturer of this pen 


Tiltlnfo 


The relationship between tilt and pen position 



a This can be combined with Manufactureld to allow the printer to download a set of 
pen characteristics. However it may be more convenient to store the characteristics 
(such as Tiltlnfo) in the pen. 



3.1 Position Tag Decoding 

3.1.1 Capture Time, Tilt and Pressure 

The pen contains sensors which allow pen tilt and nib pressure to be determined. 

3.1.1.1 Time 

All measurements are made in the context of a particular timestamp. T he pen contains a 
timer to allow equally time-spaced measurements to be made. Although we capture sam- 
ples at a rate of I 00 per second, we report time at millisecond resolution to allow for future 
improvements to the pen. Time resolution is provided at 32 bits. 
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3.1.1.2 Tilt 

The pen contains two orthogonal passive accelerometers in the x-y plane perpendicular to 
the pen axis. These respond to gravity and allow two-axis tilt to be computed to an accu- 
racy of at least 5 bits. 

Tilt is required to determine the nib position. The printer uses the two tilt values and the 
pen^s non-corrected pen position to determine the actual pen nib position. The Pen Char- 
acteristics Block contains the relationship required for the calculation. Offloading the cor- 
rection calculation to the printer reduces pen complexity and price. 

3.1.1.3 Pressure 

The pen contains a pressure sensor which measures to an accuracy of at least 5 bits. From 
the pen's point of v iew, the pressure value is only used to determine w hether the pen is on 
the paper or not. The printer is responsible for all other interpretations of pressure, such as 
determining whether an item on a page is being selected. The pen merely passes the pres- 
sure value on to the printer. 

Each position tag contains a data bit called PressureSensitiveArea. If a given pen stroke 
includes position tags that have their PressureSensitiveArea bit set, then the pressure value is 
passed as part of the stroke information. If the PressureSensitiveArea is clear, then pressure 
is implied from a stroke in terms of pen-down and pen-up. 

3.1.2 Capture the Image 

The image needs to be captured for the pen position to be determined. This only needs to 
happen if the pen is on the page. 

Io capture an image, data must be transferred from the image sensor to a fixed imasze 
bufter. This process involves a single read and write per pixel. The minimum and maxi- 
mum values encountered during the transfer are kept in variables MmPixelEncountered and 
MaxPixelEncountered, respectively, for later use in the data recovery process (see Section 
3. 1 4). 

I here are a number of general considerations that are part of the assumptions for reading 
in and decoding a position tag from an image sensor. 

With regards to the image sensor itself, there are two calculations to consider: the size of 
the sensed area, and the resolution of the image sensor. As described in Section 3. 1 .2. 1 and 
Section 3.1 .2 2, the sensed area has a required size of 193x193 dots (or 3 06mm x 
3.06mm). and the image sensor has a required resolution of 96x96 pixels. 

furthermore, as discussed in Section 3.1.2.3. the sensed image ma> be blurry and the 
sense of ambiguous macrodots mav have to be interred. 

A 96x96 image contains 92 16 8-bit pixels. We round this up to 10,000 pixels for calcula- 
tion purposes. The transfer process involves on average 6.5 cycles per pixel for a total of 
(0,000 cycles per position tag. On a 25 MM/ processor this represents 26°, b of the avail- 
able bandwidth 
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Io ensure that one entire tag is always within the w indow of the image sensor, the image 
sensor window must be the width of two tags. This requirement increases the sensed dot 
area to I 36* 136. 

Finally, since the pen must detect positional tags at any rotation, we must consider the 
worst case rotation of 45 degrees. This requirement increases the sensed dot area to 
193> 193. 

At a printed dot size of 1600 dpi, the image sensor must sense a printed area containing 
193> 193 dots, which equates to 3.06mm (0.12 inches) in each dimension. 

3.1.2.2 Image Sensor Resolution 

The basic positional tag is I 7>: I 7 macrodots. This equates to a basic sensor resolution of 
17> 17 pixels. 

To ensure that one entire tag is always within the window of the image sensor, the image 
sensor window must be the width of two tags. This requirement increases the sensor reso- 
lution to 34x34. 

Since the pen must detect positional tags at any rotation, we must consider the worst case 
rotation of 45 degrees. This requirement increases the sensor resolution to 48x48. 

Finally, to satisfy Nyquist's Theorem with respect to macrodots, we must oversample at 
least at twice the macrodot resolution. This leads to a sensor resolution of 96x96. 

Each sampled pixel is 1 byte (8 bits). We must also assume the least significant 2 bits of 
each pixel can contain noise. Decoding algorithms must therefore be noise-tolerant. 

3.1.2.3 Blurry Image 

The pen optics provide reasonable depth-of-field, and the pen includes an auto-focus 
mechanism to handle changes in pen-paper separation due to varying tilt etc., but several 

factors may still contribute to blurriness in the captured image, includine tardv auto-focus 
response, and vary ing pen-paper separation due to extreme tilt or warped paper. 




severe blurring 



Figure 9. Effect of different degrees of blurring 
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Blurring can prevent definitive macrodot recognition. Figure 9 shows the effect of blur- 
ring on a position tag composed of macrodots, ranging from no blurring to severe blurring. 
Note how isolated macrodots fade as the blurring becomes more severe. 

As the blurring increases, the more a given macrodot is influenced by the surrounding 
macrodots. Consequently the dynamic range for a particular macrodot decreases. Consider 
a "white" macrodot and a "black" macrodot, each surrounded by all possible sets of mac- 
rodots. The 9 macrodots are blurred, and the center macrodot sampled. Figure 10 shows 
the distribution of resultant center macrodot pixel values for black and white macrodots. 




Center macrodot pixel value 



Figure 10. Black-white macrodot ambiguity due to blurring 

The curve in Figure 10 is computed for an illustrative degree of blurring. The curve from 
pixel values 0-180 shows the range of black macrodots. The curve from pixel values 
75-250 shows the range of white macrodots. The greater the degree of blurring, the more 
the two curves shift towards the center of the range and therefore the greater the intersec- 
tion area, which means the more difficult it is to determine whether a given macrodot is 
black or white. A pixel value at the center point of intersection is ambiguous - the macro- 
dot is equally likely to be black or white. 



/ - r o - 



Figure 11. Relationship between data non-recovery and codeword error rate 
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scheme performs well and then substantially degrades. If there is no Reed-Solomon block 
duplication, then only I codeword needs to be in error for the data to be unrecoverable. Of 
course, with the inclusion of erasure detection (via invalid codewords), the chance of cor- 
rectly decoding the data increases. 

Figure 1 1 only links codeword errors to the total number of Reed-Solomon codewords in 
error. There is a trade-ofT between the amount of blurring that can be corrected, and the 
number of genuine errors due to dirt and paper damage that can be corrected. Since all 
error detection and correction is performed by the Reed-Solomon decoder, there are a 
finite number of errors per Reed-Solomon data block that can be corrected. The larger the 
number of errors introduced through blurring, the smaller the number of genuine errors 
that can be corrected. 



3.1.3 Locate the Position Tag 



The captured image is a 96x96 pixel map of 8-bit samples. Before any position informa- 
tion can be decoded, the position tag must be located within the image. Note that the tag 
can have anv orientation. 



The first step in locating the tag is to locate a bull's-eye target (Figure 5) within the image. 
Since we are imaging a w ider area than a single tag it is possible that two targets lie within 
the ima^e area. 
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Figure 12. Area of interest for target location 

Since an entire tag must be completely contained within the image, the data area outside 
the target must be visible. As a result, the area of interest for locating targets is actually a 
subset of the whole image. Instead of searching a 96x96 image, we only need search an 
88x88 image (a reduction of I 5°o from 92 I 6 samples to 7744). 

The 5-layer bull's-eye pattern of the target is easily found in a 2D image by scans for topo- 
logical connectivity, and is then useful for pinpointing the exact center regardless of orien- 
tation, and for determining the main axes and local x-y dimensions. 



3.1.3.1 Scan for Topofogical Connectivity 

Prior to image traversal we initialize an 88> 88 array of flags w ith 0. This array keeps the 
visited status of the corresponding pixel in the image. If the flag is 0 we have not yet vis- 
ited the pixel. If it is I or 2, then we have visited the pixel and bit I of the flag contains the 
sense of the bit (I for black, 0 for white). 
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We then start with the top left pixel and compare adjacent pixels in order to build a con- 
nectivity structure for the top left pixel. If the top left pixel is considered to be white, then 
the connectivity structure w ill represent all connected pixels that are also white. If the top 
pixel is considered to be black, then the connectivity structure will represent all connected 
pixels that are also black. As edges are met due to differences in color sense, adjacent con- 
nectivity structures are built. 

The order of pixel traversal in the image simply follows a standard pixel fill algorithm. 
There are many such algorithms to be found since this is a useful operation for paint pro- 
grams. Near-optimal algorithms such as [1] and [2] only read each pixel once on average. 

In our case we must also consider the notion of color sense. Target location does not 
require the sophistication of the macrodot reading algorithm for mode bits and data bits 
for determining if a macrodot is black or white (Section 3. 1 .4.2 and Section 3.1 .4.4). The 
minimum and maximum pixels encountered during the image capture were stored in Min- 
PixelEncountered and MaxPixelEncountered respectively, and these are used to set the thresh- 
old for black/white as MidRange = (MinPixelEncountered + MaxPixelEncountered)/2. Anything 

OrPQfAr thin S-ir- oninl t /-v UI^Dnnn^ i" r- A Anr irInr.Tf) + U. . i./UitA -1 n n .r^U^ nn 1 ^ *1 

to ' ^ ^ 1 ^- ' i"an vi «-^uui KMuivanyc 13 cuimucn.u L'tw vy miL, anu ai ly LI 1 1 i l n,5i tlldll 

MidRange is considered to be black. 

As each connectivity structures is completed, it is compared against the expected values 
for the 5 bull's-eye layers surrounding the target center. Possible matches are added to one 
list, while non-matches are discarded. 

The total time taken for processing the image is approximately 6 cycles per sample pixel. 
This equates to 46,464 cycles (88:<88:<6), or 19% of the available processor bandwidth. 

3.1.3.2 Determine Target Axes 

The connectivity structures can now be examined together in order to locate the target. 
The identification of tw o or more concentric structures is enough to determine the x and y 
axes. 

Taking one of the completely isolated concentric structures, 2 points on each of the 4 sides 
are chosen where the distance separating them is at least 3 pixels. For each point, the pixel 
of maximum whiteness or maximum blackness is chosen as the estimate of the center of 
the line. The estimate should be within I pixel of the actual center. 

The process of building a more accurate position for the line center involves reconstruct- 
ing the continuous signal for 7 scan line slices of the line. 3 to either side of the estimated 
center 1 he 7 maximum values found (one for each of these pixel dimension slices) are 
then used to reconstruct a continuous signal in the column dimension and thus to locate 
the maximum value in that dimension 
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FindMa.< { r eSamples , pos, maxVal) 
betterColumn = pos + column 



FmdMax is a function that reconstructs the original one-dimensional signal based sample 
points and returns the position of the maximum as well as the maximum value found. The 
method of signal reconstruction/resampling used is the Lanczos3 windowed sine function 
(see pages 157-158 in [6]). The function and kernel are shown in Figure 13: 
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Figure 13. Lanczos3 windowed sine function 

The Lanczos3 windowed sine function takes 7 (pixel) samples from the dimension being 
reconstructed, centered around the estimated position x, i.e. at x-3, x-2, x-l, x, x+l, x+2, 
x+3. We reconstruct points from x-l to x+l, each at an interval of 0.!, and determine 
which point is the maximum. The position that is the maximum value becomes the new- 
center. Due to the nature of the kernel, we only require 6 entries in the convolution kernel 
for points between x and x+l. We use 6 points for x-l to x, and 6 points for x to x + l, 
requiring 7 points overall in order to get pixel values from x-l to x+l since some of the 
pixels required are the same. 

Two accurate positions on each of 2 sides of the isolated bull's-eye layer give the x and y 
axes directly. Using 4 sides allows an average line to be calculated (simply the average of 
the two lines built from opposite sides). 

The distance from the center of one bull's-eye layer macrodot line to the center of the next 
bull's-eye layer's macrodot line gives the distance between macrodots in terms of a Ax and 
a Ay with respect to the current orientation of the image. 

3.1.3.3 Determine Target Center 

In order to locate the actual center of the target there are two cases to consider. 

• 1 he center of the target is one of the connectivity structures 

• The center of the target is not one of the connectivity structures 

In the first case, we choose the blackest pixel of the estimated target center, which should 
be within 1 pixel of the actual center. The process of building a more accurate position for 
the target center involves reconstructing the continuous signal for 7 scanline slices of the 
target, 3 to either side of the estimated target center. The 7 maximum values found (one for 
each of these pixel dimension slices) are then used to reconstruct a continuous signal in 
the column dimension and thus to locate the maximum value in that dimension. The pro- 
cess is the same as described in Section 3.1 .3.2 and uses the same filter as shown in Figure 
13. 



C on f ,de n! i j 



7 September 1999 



18 



Silverbrook Research 



Netpage Pen Design Description 



draft vO. 3 



If there is no current estimation for the center, the center is considered to be the calculated 
center of the four lines used in the calculation of the axes in Section 3.1.3.2. This process 
can be repeated for another isolated bulPs-eye layer and the two estimates for target center 
averaged. If the estimated center pixel is black, a more accurate center can potentially be 
obtained by following the procedure defined for the first case. However care must be 
taken since this pixel region is not completely isolated (or there would have been a target 
estimate in the connectivity structures). It may be enough to increase the accuracy in only 
one dimension. 

3.1.4 Decode the Position Tag 

With the position of the tag known, the task of decoding it can begin. Decoding a posi- 
tional tag consists of 3 essential steps: 

• Determining the orientation of the tag 

• Extracting the tag's mode data and decoding it 

• Extracting and decoding the data portion of the tag based on the mode data 

3.1.4,1 Determine Orientation 

On the four corners of the target are 3-bit clusters of orientation bits, which allow the tag's 
orientation to be quickly determined. 

Using the Ax and Ay values (obtained in Section 3. 1.3. 2), and applying them to the accu- 
rate target center coordinate (obtained in Section 3.1 .3.3), we are able to calculate the esti- 
mated positions of the 12 orientation macrodots. 

Given the macrodot coordinate (fixed point) we sample 4 image pixels to arrive at a center 

pixel value via bilinear interpolation. 

Once the center pixel value has been determined, we try to determine the bit value for that 
macrodot. To do so, we take the pixel values representing the macrodot centers to either 
side of the macrodot whose bit value is being determined, and attempt to intelligently 

guess the value of that center macrodot's bit value. Looking at the generalized blurring 
curve again (reproduced in Figure 14 from Figure 10), but with BiackMax and WhiteMm 
shown, there are three common cases to consider: 
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• The macrodot's center pixel value is higher than BlackMax, and is therefore definitely a 
white macrodot. The bit value is therefore definitely 0. 

• The macrodot's center pixel value is somewhere between BlackMax and WhiteMtn. The 
macrodot may be black, and it may be white. The value for the bit is therefore in ques- 
tion. A number of schemes can be devised to make a reasonable guess as to the value 
of the bit. These schemes must balance complexity against accuracy, and also take into 
account the fact that in some cases, there is no guaranteed solution. In those cases 
where we make a wrong bit decision, the bit will be in error, and must be corrected by 
some other means (in the case of orientation bits, the other corners may help, and in 
the case of the tag's data, the Reed-Solomon decoding stage will correct errors). 

Before the data can be extracted from the data area, the pixel ranges for "black" and white 
dots needs to be ascertained. The minimum and maximum pixels encountered during the 
search for the tag were stored in MmPixelEncountered and MaxPixelEncountered respectively. 
The following pseudocode shows the method of obtaining good values for WhiteMin and 
BlackMax based on the min 8c max pixels encountered. 



MinPixel 


= MinPixelEr 


countered 


Max P i xel 


= MaxPixelE: 


countered 


M idRange 


= (Mir.Pixei 


+ MaxPixel) / 2 


WhiteMin 


= MaxPixel - 


P e nCo nstar.tWhiteFactor 


BlackMax 


= MinPixel * 


P e n C o nst antBlack Factor 



The scheme used to determine a macrodot's value if the pixel value is between BlackMax 
and WhiteMin is fairly simple, but gives good results. It uses the pixel values of the macro- 
dot centers to the left and right of the dot in question, using their values to help determine 
a more likely value for the center dot: 

• If the two macrodots to either side are on the white side of MidRange (an average mac- 
rodot value), then we can guess that if the center macrodot were white, it would likely 
be a "definite" white. The fact that it is in the not-sure region would indicate that the 
macrodot was black, and had been affected by the surrounding white macrodots to 
make the value less sure. The macrodot value is therefore assumed to be black, and 
hence the bit value is 1 . 

• If the two macrodots to either side are on the black side of MidRange, then we can guess 
that if the center macrodot were black, it would likely be a "definite" black. The fact 
that it is in the not-sure region would indicate that the macrodot was white, and had 
been affected by the surrounding black macrodots to make the value less sure. The 
macrodot value is therefore assumed to be white, and hence the bit value is 0. 

• If one macrodot is on the black side of MidRange, and the other macrodot is on the 
w hite side of MidRange, we simply use the center macrodot value to decide. If the cen- 
ter macrodot is on the black side of MidRange, we choose black (bit value 1 ). Otherwise 
we choose white (bit value 0). 

The logic is represented by this simple pseudocode: 



(pixel < WhiteMm: ,'/ definitely black 

bit = 0x01 
e 1. s e 

if (pixel > P. lac kMa x j ,'/ definitely white 

bit = 0 x 0 0 
else 
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if ((prev > MidRange) 4 & (next> MidRange)) //prob black 

bit = 0x0 1 
else 

if {(prev • : MidRange) (next < MidRange}) //prob white 

bit = 0x00 
else 

if (pixel MidRange) 

bit = 0x01 
else 

bit - 0x00 



From this one can see that using surrounding pixel values can give a good indication of the 
value of the center macrodot's state. The scheme described here only uses the macrodots 
from the same row, but increased accuracy may be achieved using all the neighboring 
macrodots, at the cost of increased complexity. 

We already have the distance between the macrodots in terms of a Ax and a Ay with 
respect to the current orientation of the image. Rather than rotate the image to transform 
Ax and Ay both to 1, we simply change the sense of the Ax and Ay values based on how 
many sets of 90 degrees it takes for the tag to be oriented so that both D values are posi- 
tive. The four possible rotations by 90 degrees equate to 4 different combinations of signs 
(±Ax and ±Ay). It is therefore trivial to translate Ax and Ay into two deltas for movement in 
x and y: Ax x and Ay x for movement of 1 macrodot in x, and Ax y and Ay y for movement of 1 
macrodot in y. 

With the 12 orientation bits obtained, the orientation of the tag can be easily determined. 
Given 4 sets of 12 bits, the correct orientation is the one that minimizes the number of bits 
in error, as shown in Table 5. 



Table 5. The four major orientations of the position tag 



12-bit value 


Orientation of tag 


111-011-100-000 


No rotation 


011-100-000-111 


Rotated clockwise 90 degrees 


100-000-111-011 


Rotated clockwise 180 degrees 


000-111-011-100 


Rotated clockwise 270 degrees 



3.1.4.2 Extract the Tag's Mode Raw Data 

With the orientation determined, the next step is to read the bit pattern representing the 
Mode bits from the tag. 

We know the coordinate of the tag's target center as [Center x . Center y j. We also know the 
orientation of the tag and therefore have Ax x and Ay x for motion in x, and Ax y and Ay y for 
movement in y. These delta values enable us to step along the center of any macrodots. By 
sampling the tag at the 18 mode bit positions (see Figure 6). Consequently the location of 
the first few mode bit are: 

• BitO location - [Center x - 5Ax x - 5Ax r Center y - 5Ay x - 5Ay y ] 

• RiH location [RitO, * Ax,. Bit0 T . *Av r ] 
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The extraction of 28 bits will take approximately 168 cycles on a simple microprocessor. 
Assuming a clock speed of 25 MHz, 168 cycles per position sample represents less than 
0. 1 % of the available processor bandwidth. 

3. 1.4.3 Decode the Mode Data 

The 28-bit raw bitstream representing the tag's mode data has been read in. It must now be 
decoded into 8 bits of meaningful mode information. 

The data is decoded using straightforward Reed-Solomon decoding. One of the advan- 
tages of using Galois Field GF(16) is that a total of only 64 bytes are required for lookup 
tables during the decoding process (16 bytes each of power, log, multiply and multiply 
inverse). 

The decoding process of 28 bits to 8 bits will take approximately 1 500 cycles on a simple 
microprocessor. Assuming a clock speed of 25 MHz, 1500 cycles per position sample is 
less than 1% of the available processor bandwidth. 

3.1.4.4 Extract the Raw Tag Data 

Before the tag data proper can be read in from the tag image, the tag's 8-bit mode data 
must be examined. The Version bit must be checked to ensure that this tag has a known 
structure (see Table 3). If the Version bit is 0, then the tag data structure is known and can 
be read m from the image. 

The macrodots representing the tag data area are read into a 168-bit raw bitstream accord- 
ing to the locations described in Figure 7. We use the tag's target center [Center x , Center y ] 
and the orientation delta values Ax x and Ay x , Ax y and Ay y to calculate the start position. The 
delta values also enable us to step along the center of the data area's macrodots. 

The process of determining if a given macrodot is a 1 or a 0 is the same as that carried out 
for reading the tag's mode bits and orientation bits (see Section 3. 1 .4. 1 ). A total of 168 bits 

is read in from the image in this way. The extraction of 168 bits will take approximately 

1008 cycles on a simple microprocessor. Assuming a clock speed of 25 MHz, 1008 cvcles 
per position sample represents 0.4% of the available processor bandwidth. 

3. 1 .4.5 Decode the Tag Data 

The 168-bit raw bitstream representing the tag data has been read in. It must now be 
decoded into the Pageld, x and y coordinates etc. (see Table 2). 

The 28 x 6-bit codewords can be divided into message data codewords and check words. 
Assuming that the Version bit is 0, the 2-bit NumMsgWords field in the Mode area deter- 
mines the number of message data codewords D as 15. 16, 17, or 18. The remaining K 
codewords are check codewords. 

Fach of the D 6-bit message codew ords can now be examined to see it it is ail 0s or all 1 s. 
Any such codeword is illegal (due to the encoding process) and can be considered to have 
been erased. Determining that a codeword has been erased increases the error correction 
capabilities of the check bits. 

The data can then be decoded using straightforward Reed-Solomon decoding One of the 
adv antages of using Galois Field GF(64) is that a total of only 256 bytes are required for 
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lookup tables during the decoding process (64 bytes each of power, log, multiply and mul- 
tiply inverse). 

The decoding process of 168 bits to 90 bits will take approximately 4000 cycles on a sim- 
ple microprocessor. Assuming a clock speed of 25 MHz, 4000 cycles per position sample 
represents 1 .6% of the available processor bandwidth. 



3.1.5 Build Stroke Information 



The information gained from the position tag is either ignored or added to the current 
stroke depending on whether the pen is deemed, based on nib pressure, to be up or down. 
If the pen is up but was down, then the current stroke is finished. If the pen was up but is 
now down, then a new stroke is started. 

Each stroke contains the start time (in milliseconds) followed by the recovered page id and 
initial position. This is followed by a series of positions for the stroke. The positions are 
implicitly separated by lOOths of a second in time. While the first position is always abso- 

, F v^i«..^uj uil ^imci ucud-ciiLuucu <ji aosoiuie as required, tscape codes 

allow compression for a number of unknown positions (the tag cannot be found), the 
encounter of different page ids (for example the initial page id is unknown and finally a 
page id is recovered from a tag during the stroke, or it may be that the pen crosses from 
one page to another during a stroke), and the encountering of tags with differing Pressure- 
SensitiveArea values, which enable or disable the inclusion of a pressure value with each 
position. Pressure values are typically included for the entire stroke if the stroke begins in 
a pressure-sensitive area. 

The following is a definition for a stroke: 



Stroke - time PageldRee FirstPosRec ([PosRec I PressurePos I Escape]} endStroke 
Page idRec - [ u r. k n o w r. ! p a g e I d ] 

Escape = [pa geld I p ressureMode I TimeSkip] 
TimeSkip = [unknown | zeroDelta] shortTime 

PressurePos = PosRec penPressure 
PosRec - [Per.AcsRe: : PenDe 1 t a Re c ] 

PenAbsRec = t a ;Coc r d ; r.a ■ e i ma geCo o r d i r. a t e rr.ac r odo t De 1 t a tilt 
PenOeltaP.ee = penDelta 1 mageCoo rd 1 na t e mac r odo t De 1 1 a tilt 



time - 32 bit.i 



. i lseconds ) 



pen Dei ta - 2 x 4 b : t s signed (representing -8.-7 for x am: y ) 
penPressure = 5 bits 

tilt - 2 x 5 bits : represent mg 2 tilt axes; 
P res -.5' j re Mode -= 1 fc : - 



The length of a sample position is 64 bits (18 + 20 + 16 + 10) when the pen coordinate is 
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For a stroke over a single page that lasts 4 seconds, and assuming no read errors, there is 
an initial pen down followed by 399 deltas. The total stroke length is therefore 21,708 bits 
( 1 62 + (399 x 54)), which is 27 1 4 bytes, or 6.8 bytes per sample. 

If pressure is also captured with each position (depending on the value of the encountered 
PressureSensitiveArea bits within each tag), a further 5 bits are required for each position. 
The Escape of pressureMode allows the toggling of this state within a stroke. 

The stroke definition copes well with unknown pen locations. If the initial position 
(pageld, coordinates etc.) is unknown due to the inability to locate a tag or recover tag 
data, the initial position for the stroke is marked as unknown. This initial position may be 
followed by an Escape of TimeSkip, signifying the amount of time where unknown posi- 
tions were captured. Finally, when a tag is captured and decoded successfully, the Escape 
for pageld is used. This pageld is valid until the next Escape pageld is included in the stroke 
(inserted when a tag is encountered with a pageld different to the current known pageld). If 
a tag cannot be found or decoded successfully mid-stream within a stroke, the Escape for 
TimeSkip is used, which represents the elapsed time where only unknown positions were 
sampled. An unknown pen position does not change the pageld. If tag positions are reac- 
quired, the page id may be changed, depending on the tag. A Timeskip entry for a set of 
unknown positions uses 1 bit for the escape code, 2 bits to define the type of escape code, 
and 16 bits to define the duration of samples skipped. 

The other use for the Escape of TimeSkip is when there have been one or more deltas of 0. 
The Escape of zeroDelta combines with a 16-bit time amount to specify how long the pen 
has remained in the same position without moving. 

The pen contains 0.5MB of RAM to buffer stroke information both during stroke con- 
struction and if the pen is out of range of the printer. A 0.5MB buffer allow s for approxi- 
mately 192 strokes at 4 seconds per stroke, which represents more than 12 minutes of 
stroke capture. The exact duration will depend on the success of tag decoding, and 
whether or not continuous pressure is being captured. 

If the stroke buffer becomes full, a Buffer-Full LED is illuminated. The LED stays illumi- 
nated until the pen comes into range of a printer, w hereupon the buffered strokes are trans- 
mitted to the printer and the buffer freed. 



The position tag data area contains a ActiveArea bit (see Table 2). The pen's active area 
LED must be enabled directly in response to the value of this bit. T here is no additional 
interpretation of the ActiveArea bit by the pen. 



The data stream must be encrypted before being transmitted to the printer, to prevent 
eavesdroppers from capturing pen input, and particularly signature input 

Consequently, the final stage before transmission is the encryption of the data stream. 
Symmetric encry ption is used since it is tractable in the time available. Possible symmetric 
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algorithms for use are listed in Table 6. In all cases, a minimum key size of 14 bytes is 
assumed. 



Table 6. Symmetric encryption algorithms [5] 









^^^^^^^ 






Blowfish 


Block 


18 


8 


4-56 


No 


CAST 


Block 


20 


8 


16 


Yes, but free 


IDEA 


Block 


50 


8 


16 


Yes 


Tnple-DES 


Block 


108 


8 


14 a 


No 



a Although Tnple-DES uses a 168 bit key. the effective key strength is only 112 bits 



Of the algorithms listed in Table 6, Triple-DES is the most conservative choice, although 
it is the slowest. At 108 clocks per byte processed, the encoding of a single 8-byte block 
consumes 864 cycles (8 x 108V 

The pen must send 100 encrypted samples each second. Assuming that each sample 
requires a complete Triple-DES block (8 bytes), the time taken to encrypt 1 second of data 
is 86,400 cycles. 

For a clock speed of 25 MHz, 86,400 cycles for the 100 samples equates to less than 0.5% 
of the processor bandwidth. 

The encry ption process therefore has a throughput of 800 bytes per second. 



Silverbrook Research 



Netpage Pen Design Description 



draft vO. 3 




Confident), il 



7 September 1999 



26 



SMverbrook Research 



Netpage Pen Design Description 



draft vO. 3 



4 Silverbrook References 



Silverbrook Research 



Netpage Pen Design Description 



draft vO. 3 



5 Other References 

[1] Fishkin, K., "Filling a Region in a Frame Buffer", Graphics Gems, Academic Press 1990, pp.278-284 
[2] Heckbert, P.S., "A Seed Fill Algorithm", Graphics Gems, Academic Press 1990, pp.275-277 
[3] Lyppens, H., "Reed-Solomon Error Correction", Dr. Dobb's Journal Vol.22, No.l, January 1997 
[4] Rorabaugh, C, Error Coding Cookbook, McGraw-Hill 1996 

[5] Schneier, S., "The Crypto Bomb is Ticking", Byte Magazine, May 1998, pp.97- 102 
[6] Turkowski, K., "Filters for Common Resampling Tasks", Graphics Gems, Academic Press 1990, 
pp. 147-165 

[7] Welch Allyn Inc., Aztec Barcode Symbology Specification Rev 3.0, 1995 

http: / /dcd . we lcha 1 lyn . com/ 1 echo ve r/dcdwhite .htm 
[8] Wicker, S., and Bhargava, V., Reed-Solomon Codes and their Applications, IEEE Press 1994 



Confidential 



7 September 1999 



28 



Netpage 

Extensions 

draft version 0.2, 26 November 1999 



Silverbrook Research Pty I td 



( ax + 61 2 9818 6711 

Email: info@silverbrook.com.au 

Confidential 



draft vO 



Document History 



Version 


Date 


Authors 


Details 


0.2d 


25 November 1999 


Paul Lapstun 


Added descriptions of payment card reader in 






Kia Silverbrook 


printer, vacation mode for printer, page-driven 






Simon Walmsley 


marking mode and color, long-range sensor 








with laser pointer, television control via 








Netpage, and telephone control via Netpage. 








Expanded description of hybrid electronic ink 








Netpages. 








Added descriptions of magic lenses 


Old 


20 October 1999 


Paul Lapstun 


Initial draft issue 






Kia Silverbrook 








Simon Walmsley 





26 November 1 999 



Silverbrook Research 



Netpage Extensions 



draft vO 2 



Contents 

Netpage Extensions <\ 

1 Overview 2 

2 Netpage Printer 3 

2.1 Braille Output 3 

2 2 Printer Embedded in Digital Television Set 3 

2.3 Payment Card Reader in Printer 3 

2.4 Vacation Mode for Printer 3 

3 Netpage Pen / Sensor 4 

3.1 Universal Drawing Implement 4 

3.2 Output via LCD 4 

3.3 Output via LED 5 

3 4 Olltnilt via Tr\m\nr I PPi 

- - -r--* — — ~ b 

3.5 Interchangeable Nibs 5 

3.6 Audio Output 5 

3 7 Language Translation Output 5 

3.8 Page-Driven Marking Mode and Color 5 

3.9 Long-Range Sensor with Laser Pointer 7 

3.10 Television Control via Netpage 7 

3.11 Telephone Control via Netpage 7 

3.12 Physically Erasable Digital Ink 8 

4 Electronic Ink g 

4.1 Reusable Electronic Ink Netpages 9 

4.2 Selectively Writable Electronic Ink Netpages 9 

4.3 Hybrid Color and Electronic Ink Printed Netpages 9 

5 Magic Lenses 10 

5 1 Audio Clip Magic Lens 10 

5.2 Video Clip Magic Lens 10 

5.3 Text-to-Speech Magic Lens ^ 

5.4 Netpage Magic Lens ^ 

5.5 Netpage Printing Magic Lens ^ 

5 6 Braille Magic Lens 12 

References ^3 

6 Silverbrook References... 14 

7 Other References 15 



Silverbrook Research Netpage Extensions 



draft vO 2 



Netpage 
Extensions 



26 November 1999 



1 



Iverbrook Research 



Netpage Extensions 



draft vO 2 



1 Overview 

This document describes a set of possible extensions to the core Netpage architecture. 
These extensions are described separately from the core architecture to ensure that the 
core architecture is clearly understood. 
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2 Netpage Printer 

2.1 Braille Output 

A Netpage Printer can optionally contain a stamping mechanism for imprinting Braille 
characters on paper. The printer can thus produce an interactive Netpage which also con- 
tains the textual content of the page encoded in Braille. Documents delivered to the printer 
are personalized for the user in the usual way so that the text size and font match a useful 
Braille character size. Each sheet of paper is printed on one side only. Graphic elements on 
a page, such as images and buttons which do not contain text, are tagged with text in 
Braille. A Braille Netpage functions in the same way as a normal Netpage. 



2.2 Printer Embedded in Digital Television Set 

A Netpage Printer can be embedded in a digital television set and receive data from the 
Netpage Network via a digital television ^'datacasr channel. It can use a low-bandwidth 
mobile telephone or pager channel to send data to the network. The printer thus leverages 
the communications capability of the digital television set, and provides a compelling use 
of a datacast channel. It also provides opportunities for seamlessly linking broadcast tele- 
vision content, such as infotainment, with printed material, as described in Section 9 2 1 in 
[11- 



2.3 Payment Card Reader in Printer 

A Netpage Printer can optionally include a payment card reader for reading account infor- 
mation from a payment card as part of an payment transaction. The reader can support 
magnetic stripe cards, smart cards, etc. The printer may contain a keypad to allow the user 
to enter a personal ident in cat ion number (PIN) in conjunction with the card. 

The payment card reader and keypad can be provided as an optional module which con- 
nects to the printer via a flexible cable and mounts in a bracket on the side of the printer or 
on the wall when not in use. It can also communicate with the printer using the printer's 
standard local uireless communications mechanism which is also used b\ the pen. 



2.4 Vacation Mode for Printer 

A Netpage Printer can be placed in 'vacation 1 mode, where documents addressed to the 
printer are not printed, but are simply persistently stored on the Netpage Network as usual. 
When the printer is taken out of vacation mode, a list of pending documents is automati- 
cal^ printed. Pach entry in the list describes the corresponding document, indicates the 
date and time it was delivered, and contains a <print ■ button for the document. A separate 
list is printed for each user with documents pending on the printer. 

Vacation mode can be set and reset via the printer's help page. To ensure that vacation 
mode is not accidentally left set, however, the printer can contain a prominently positioned 
and illuminated vacation button. The vacation button toggles the printer's vacation mode, 
and its illumination clearly indicates whether the printer is in vacation mode or not. 
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3 Netpage Pen / Sensor 

3.1 Universal Drawing Implement 

A small Memjet printhead can be used to construct a universal drawing implement. A 
small cylindrical roller is in contact with the page. The roller spins freely about the axis of 
the stylus, so that as the roller is dragged across the page, it automatically orients itself so 
that it is at right angles to the direction of motion. The Memjet printhead is mounted 
behind the roller. It prints onto the back of the roller, and the roller transfers the printed 
image onto the page. A small cleaning station cleans any ink off the surface of the roller 
after it has contacted the page, so that the printhead always prints onto a clean surface. The 
printhead reproduces a contone color image via dithered bi-level CM Y or CMYK. The 
stylus can be programmed to produce any colored, textured continuous line or paintstroke 
The rate of printing is dictated by the speed of movement of the roller relative to the pa«e 
This can be determined in several ways, e.g. from the actual rotation of the roller, or by 
continuously 

^ D---C? u.,^ ^i^iiu^ niu vciuciii iii me same way as a sec- 

ond-generation optical mouse, or by sensing and decoding the map of a self-mappin° sur- 
face. & 

In an alternative implementation the Memjet printhead prints directly onto the pat>e rather 
than onto a roller. 

The stylus can also be switched into non-marking mode, obviating the need for both mark- 
ing and n on- marking nibs. 

The user can load virtual colors, textures and line styles into the universal stylus from 
printed palettes. The stylus can optionally indicate its current settings via an LCD or 
LLDs. as described below in Section 3.2, Section 3.3 and Section 3.4. 

3.2 Output via LCD 

A Netpage Pen or similar sensing device can optionally incorporate a LCD which can be 
used to display modal, selection, an status information: 

• mode name 

• current color/texture for drawing paint inn 

• current line width for drawing 

• current brush for painting 

• first few words of current text selection 

• image fragment of current image selection 

• URI of hyperlink selection 

• pen status (power, communication with printer, etc.) 

• error messaues 



I his allows immediate feedback without requiring an addition;!! trwe to he 
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3.3 Output via LED 



A Netpage Pen or similar sensing device can optionally incorporate an UiD which can be 
used to indicate that the pen is in a mode. The LPD can server as a reminder that the user 
recently selected a non-standard mode, e.g. a colored brush for painting. An on-page but- 
ton can be used to clear the current mode. 



3.4 Output via Tricolor LED 



A Netpage Pen or similar sensing device can optionally incorporate three LKDs - one red. 
one green, and one blue - which can be used in combination to indicate a continous-tone 
color currently associated with the sensing device, for example a virtual pen or brush color 
when the sensing device is being used to interact with a drawing or painting application. 



3.5 Interchangeable Nibs 



A Netpage Pen or similar sensing and marking device can optionally support interchange- 
able nibs. Nibs can have different shapes, producing different line styles, and can have dif- 
ferent colored ink supplies built m, producing different line colors. They can also produce 
diHerent textures, depending on the composition of the nib. f-ach nib contains a code 
which uniquely identifies the type of the nib. The pen obtains the nib's identity code when 
the nib is attached to the pen. Any digital ink captured by the pen while the nib is attached 
is labelled with the nib's id 

Since some nibs will produce a different effect depending on their tilt and rotation, tilt and 
rotation must be be embedded in the digital ink in such cases. Since the Netpage Pen is 
typically used in a somewhat fixed rotation, the nib rotation may be a function of how the 
nib is attached to the pen 

I he nib id embedded in the digital ink can then be used b> Netpage software to label the 
digital ink with line, color and texture characteristics which mimic those of the physical 

nib. A user can also choose to map a particular nib to an arbitrary pen or brush style, for 
example an air brush. Although the strokes reproduced by the Netpage software will no 
longer match the strokes produced by the physical nib. the user is able to distinguish 
strokes of different kinds on the physical page. 

The nib id can also be used to signal the intent of the strokes produced with the nib. A par- 
ticular nib might be designated as a text editing nib. for example, used solely to mark up 
text. By dedicating a nib to a particular function, it becomes easier for the software to dis- 
ambiguate data {e.g. text) and commands (e.g. text editing commands). 

An\ number of Netpage Pens registered to the same user can be sem i-permanently dedi- 
cated to particular drawing styles or functions by having function-specific nibs semi-per- 
manently attached. 



3.6 Audio Output 



When a user interacts u ith a Netpage when out-of-range of a Netpage Printer, dmital ink 
is buttered in the pen and is transmitted to and interpreted by the printer when the pen later 
comes w ithin range of it. As a way of obtaining immediate feedback, pages can contain 
audio hyperlinks. When the user presses an audio hyperlink, the pen communicates with 
the user's mobile phone, the phone obtains the audio clip from the Netpage Network, and 
the phone plays the audio clip to the user. This assumes the pen and the phone share a 
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short-range wireless technology such as Bluetooth, and that it is economical for the mobile 
phone to have an open data connection to the network such as promised by third-genera- 
tion mobile telephony. It works best if the user is using the phone in hands-free mode 
together with a wired or wireless (Bluetooth) earphone. 

The pen can also contain its own mobile telephony module and audio output, thus elimi- 
nating the need to interface with a mobile phone. Similarly, if audio output is restricted to 
only operate when the pen is within range of a printer, then only an audio output capability 
is required, either in the printer or in the pen. 

3.7 Language Translation Output 

The Netpage System can offer an automated translation service. The user selects a word, a 
sentence, paragraph, or arbitrary block of text, and then presses a <translate> button, per- 
haps on the help page. The translation is displayed on the LCD, as described in Section 
3.2. If the translation is too long to be displayed all at once, then it scrolls horizontally, 
slowly enough to be read. 

Alternatively, the translation is delivered by audio output as described in Section 5. 1 and 
Section 5.3. 



3.8 Page-Driven Marking Mode and Color 

In a pen which supports automatic switching between marking and non-marking modes, 
the current mode can be dictated by the current area of the page with which the pen is 
interacting. Thus the pen can automatically switch to marking mode when interacting with 
areas on the page which accept input, such as drawing fields and text fields, and can auto- 
matically switch to non-marking mode when interacting with areas on the page which 
don't accept input, such as buttons and hyperlinks, and areas of static information which 
can be selected but not modified, such as printed text and images. 

If the page is a self-mapping surface, then each surface tag can contain a control bit which 
informs the pen whether the tag is in a marking or non -marking area. Controlling the 
marking mode at tag resolution is perfectly adequate. Higher-resolution control is possible 
it the pen queries the page description via the printer, although this is likely to lead to 
unacceptable latency. 

In a pen which supports automatic switching between multiple marking modes, such as 
different colors, line st\ les and textures as described in Section 3. 1 above, then the current 
mode can also be dictated by the current area of the page with which the pen is interact inu. 
I'hus the pen can automatically switch to a black line mode uhen in a text field, for exam- 
ple, but allow user-selectable colors and styles in a drawing Held. 

If the page is a self-mapping surface, then each surface tag can contain multiple control 
bits which inform the pen of the marking mode of the area One control value can corre- 
spond to a non-marking mode. Another control value can correspond to 'don't care', 
allowing the user to use a selected mode rather than a mode dictated bv the pa»e. 
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3.9 Long-Range Sensor with Laser Pointer 

A user can interact with a Netpage at a distance using a long-range sensor coupled with a 
laser pointer. This can be useful, for example, for following information links embedded 
in printed advertising appearing in public places, where it may be inconvenient to interact 
physically with the page. 

The long-range sensor can capture an infrared image of the field of view centered on the 
laser pointer axis, and can then recognize a small number of large tags in the image which 
are printed on the page. A page can be printed with tags at multiple scales to aid recogni- 
tion at different viewing distances. The sensor should also ideally recognize the position 
of the laser spot in the image, to avoid having to rely on precise knowledge of the spatial 
relationship between the laser pointer axis and the image capture axis. The tags identify 
the page and the scale at which they are defined, and allow the size and orientation of the 
page w ithin the image to be accurately determined. This in turn alows the position of the 
laser spot in the image to be translated into a position on the page. A button click on the 
sensing device can then be interpreted as a click on the page at the position of the laser 
spot, which is then handled in the normal way. 

The sensing device communicates with a Netpage Printer in the usual way of a Netpage 
Pen, and buffers button clicks when out-of-range of a printer. 

3.10 Television Control via Netpage 

A Netpage can be used to control a television receiver. VCR or video-on-demand unit. 
I he Netpage can contain printed control buttons in the style of a remote control unit, or 
printed playback control buttons associated with individual programs or movies. Playback 
controls can include play, pause, stop, start and end. An iconic image of the program or 
movie can itself act as a play button. 

Consumer disk-based video recorders are now available which support simultaneous 
recording and playback. They allow buffered viewing of broadcast video and provide 

some of the advantages of video-on-demand. A daily newspaper delivered v ia the Netpaize 
Svstem can contain links to broadcast new s video captured in this w ay just before delivers 
ot the new spaper, perhaps under control of the Netpage Printer. 

With broadcast television, a printed Netpage program listing can provide a useful interface 
to a collection of television channels, particularly if one or more channels are being buff- 
ered by a disk-based video recorder. With true video-on-demand, a printed Netpage cata- 
log can provide a useful interface to the video I i bran.-. 

I he Netpage Pen can be augmented to work as an infrared remote control. Commands 
generated by pen impactions with printed video playback controls can also be sent 
directly from the Netpage Printer to the video device, by wired or wireless control inter- 
lace, including infrared and radio (e.g. Bluetooth). 

3.11 Telephone Control via Netpage 

A Netpage can be used to control a fixed or mobile telephone. The Netpage can contain 
printed control buttons in the style of a telephone, or printed 'calf buttons associated with 
contacts in a contact list or on a business card 

Commands generated by pen interactions with printed telephone controls can be sent 
directly from the Netpage Printer to the telephone device, by wired or wireless control 
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interface, including radio (e.g. Bluetooth). Commands can also be sent to a PABX, or to a 
control node on a mobile telephony network. 

In the case of a mobile telephone, the telephone can provide the pen with a data link to the 
Netpage Network, allowing pen interactions with a Netpage to be resolved even while the 
pen is out-of-range of a printer, as described in Section 3.6 above. If the Netpage sensor is 
built into the mobile telephone, then the need for pen-telephone communication is elimi- 
nated, and the user is able to dial a number by simply pressing on a printed 'calf link with 
the telephone. 

3.12 Physically Erasable Digital Ink 

Dye-based inks can be erased with bleaching agents which are specific to classes of dyes 
or individual dyes. A Netpage Pen can be augmented with an erasing nib, perhaps 
mounted at the opposite end of the pen to the marking nib. A user can use the erasing nib 
to apply a bleaching agent to the page and thus erase any marking ink on the page. While 
the erasing nib is being used the pen captures digital ink in the usual way and tags it as era- 
sure, so that the digital ink attached to the virtual page instance can be erased in registra- 
tion with the physically erased ink. If erasure is supported, digital ink must be retained 
even when it is subsequently converted to other forms such as text, so that the conversion 
process can be re-performed on the edited digital ink. Digital ink in a signature field, how- 
ever, should not be retained, nor should physical ink in a signature field be erasable. 

Marking inks and bleaching agents may be co-designed so that the marking inks are only 
erasable within a fixed time period of application to the page. Thus the erasing nib can 
only be used to correct recent mistakes, not to edit long-standing input. Digital ink can 
then be discarded from the virtual page instance once it reaches a certain age. If the pen 
supports a page-driven marking mode, as described in Section 3.8 above, then this can 
also be used to prevent the erasing nib from attempting to erase input which is no longer 
erasable as indicated by the age of its digital ink. 

Rather than using bleach, the erasing nib can use a highlight color which indicates, by 

convention, that the underlying ink has been erased. If a page containing such erasures is 
re-printed, then the erasures can be seen to have taken effect. 
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4 Electronic Ink 

Various "electronic ink" technologies are emerging, including E Ink's electronic ink [2] 
and Xerox' Gyricon electronic ink [4]. Both of these technologies involve bistable mono- 
chrome dots. They can be addressed externally, e.g. by passing them through a "printer 11 , 
or internally, by including addressing circuitry for each dot. There are several advantages 
to electronic ink. It has a display quality approaching that of ink on paper, since it is reflec- 
tive rather than emissive or transmissive. It can be rewritten any number of times, just like 
a normal computer display, and arguably at fast enough rates to allow animation. And it is 
inexpensive enough be bound into multi-sheet books, giving multi-sheet navigability just 
like paper books. 

In the short term it may only be practical to address it externally, and thus its main advan- 
tage may only be that it is easily reusable. 

4.1 Reusable Electronic Ink Netpages 

A reusable electronic ink page can be made Netpage addressable. Once "printed" onto an 
electronic ink page, the page can have all the functionality of a normal Netpage. What is 
required, however, is to introduce a level of indirection between the Netpage page id and 
the logical page, since the "content' 1 of the physical page can change. Thus whenever the 
page is referenced by the pen, the tags are translated to the page id of the current Netpage 
displayed on the electronic ink page. 

4.2 Selectively Writable Electronic Ink Netpages 

An electronic ink page can be made writable with the Netpage Pen. When the pen is 
moved across the page in marking mode, i.e. when marking is enabled and pen pressure 
indicates "pen down", the pen can electrostatically flip electronic ink dots on. When the 
pen is moved across the page in erasing mode, the pen can electrostatically flip electronic 
ink dots off. If the page is made Netpage addressable, regions of the page can be desig- 
nated as non-markable via a flag bit in the surface tags. In a non-markable region the pen 
won't modify dot states. 

4.3 Hybrid Color and Electronic Ink Printed Netpages 

Data can be impressed on an electronic ink page both via its electronic ink dots and via 
normal dye-based or pigment-based colored ink dots on its surface. Whereas the electronic 
ink dots can be made modifiable as described above, normal ink dots can remain un-mod- 
ifiable even where they coincide with electronic ink dots. This allows modifiable informa- 
tion to be pre-printed together with un-modifiable information. It also allows markings 
made by the user to be modifiable without affecting un-modiflable pre-printed informa- 
tion. Thus a user can erase an incorrectly entered value in a text box without erasing the 
boundary' or even the background of the text box. The electronic ink page can be made 
Netpage addressable via information printed in the same way as and at the same time as 
the normal ink dots, removing the need for a level of indirection between the page id and 
the page contents as discussed above. 
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5 Magic Lenses 

A 'magic lens 1 is a device which, when placed on top of some kind of information, pro- 
vides a different view of that information [2]. In the Netpage context, a magic lens is a 
purpose-specific hardware device which is used to provide a different view of information 
on a printed Netpage. Although Magic Lens is a trademark of Xerox Corporation, it is 
used as a generic term in the computer graphics community. 

5.1 Audio Clip Magic Lens 

An audio clip magic lens is a small device which contains a Netpage sensor, a transceiver 
for communicating with the base station, an audio digital-to-analog converter, a small 
audio amplifier, a speaker, a stereo headphone socket, a volume control potentiometer, and 
a controlling processor. 

It typically contains playback controls such as play, pause, fast forward, rewind, start and 
end, although these can also be provided in printed form on the Netpage. It typically also 
contains a small segment LCD for displaying audio clip information, including clip'dura- 
tion, elapsed time, and play/pause status. 

When the user touches the audio magic lens to a audio clip link on a Netpage, the magic 
lens senses the page via its sensor, and decodes the link in the usual way with the help of 
the base staion. It then streams the audio from the Netpage Network via the base station, 
subject to the user's interaction with the playback controls. 

Audio is routed to the speaker, or, if a plug is present in the stereo headphone socket, to the 
attached headphones. Audio may also be routed to wireless headphones via the trans- 
ceiver, either directly from the base station or via the magic lens. 

I he audio magic lens may operate out-of-range of the base station if it is enabled to 
exploit mobile telephony as described in Section 3.6 above. 

5.2 Video Clip Magic Lens 

An video clip magic lens is a small device which contains a Netpage sensor, a transceiver 
for communicating with the base station, an audio digital-to-analog converter, a speaker, a 
stereo headphone socket, a volume control potentiometer, a color display, a displa> con- 
troller, and a controlling processor. 

It typically contains playback controls such as play, pause, fast forward, rewind, start and 
end. although these can also be provided in printed form on the Netpage. It uses the color 
display for displaying video clip information, including clip duration, elapsed time, and 
play/pause status, optionally superimposed on the video output. 

When the user touches the video magic lens to a video clip link on a Netpage, the mauic 
lens senses the page via its sensor, and decodes the link in the usual way with the help of 
the base staion. It then streams the video from the Netpage Network to the display via the 
base station, subject to the user's interaction with the playback controls 
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The Netpage sensor in the video magic lens may be configured in such a way that the user 
can simply place the magic lens somewhere on a Netpage, and the magic lens will display 
video in any video region under the magic lens, registered to the region. 

5.3 Text-to-Speech Magic Lens 

The text-to-speech magic lens is a version of the audio magic lens which, instead of play- 
ing back an audio clip, plays back the text under the lens as speech. It can also perform 
language translation. 

5.4 Netpage Magic Lens 

The Netpage magic lens is a generalization of the audio and video magic lenses. It dis- 
plays a registered image of the part of the page which it covers, and makes audio and 
video directly available via on-screen controls. The screen itself can be Netpage-encoded 
so that the Netpage Pen can be used to interact with the screen in the same way as it is 
used to interact with the printed Netpage. All interactive features of the underlying page 
remain active on the screen of the magic lens. The screen can also be touch-sensitive so 
that interaction via a passive stylus is possible, including the Netpage Pen. 

Controls which provide access to interactive features are typically only shown when the 
page is viewed 'through' the magic lens. They may be shown on the printed page as well, 
although ideally in a form which clearly indicates that they are inactive. Status informa- 
tion such as elapsed time is also only shown on-screen. The underlying page layout 
should, however, provide space for the display of interactive controls and status informa- 
tion, so that the printed page and on-screen display remain compatible. 

Interactiv e control is not limited to playback of streaming media such as audio and video. 
It can also include, for example, manipulation of an object in 3D. 

The Netpage magic lens can be transparent when it is not in contact w ith a page, so that 
while it is being placed on a page, the page is seen through the lens. This can be imple- 
mented using a LCD shutter which is reflective when opaque, positioned behind a trans- 
missive color display. However, the Netpage magic lens design is simplified if the lens is 
always opaque, and only ever * virtually 1 transparent. This has the added benefit that when 
the lens is lifted from the page the portion of the page with which the lens was last in con- 
tact is retained by the lens and remains interactive. 

The Netpage magic lens is configured so that its Netpage sensor senses the position and 
orientation of the lens on the page. The lens can contain multiple Netpage sensors (one in 
each corner, for example), so that it works when only partially overlapping a page, or even 
w hen overlapping multiple pages. One sensor located in the center of the lens is sufficient 
for most purposes, however. 

The Netpage magic lens obtains and renders relevant page data from the Netpage Printer. 
The Netpage magic lens may double as a personal digital assistant. 

5.5 Netpage Printing Magic Lens 

The Netpage magic lens can be augmented with a lens-width (or tens-height) scanning 
Memjet printhead which physically copies onto the underlying page any virtual markings 
made by the user via the lens display. Virtual strokes can be faithfully reproduced, includ- 
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ing colors, textures, line widths, and even subtle brushing effects such as produced by stri- 
ated brushes and airbrushes. A printing magic lens must include a replaceable color ink 
cartridge to supply the Memjet printhead with ink. 

In the limit case, a user interacts with Netpages exclusively via the Netpage printing magic 
lens, eliminating the need for the Netpage Pen. 

Although a Memjet printhead provides 1600 dpi resolution, a printing magic lens can get 
by with less. If the printing magic lens is sufficiently small, then an area printhead can be 
used rather than a scanning printhead. 

The Netpage printing magic lens can also be used to paste information onto a page, 
including images and text. The user may take a photo with a camera and paste it into an 
image field requiring the photo. The printing magic lens links the photo to the image field 
and faithfully transfers the photo to the page, registered with the image field. If the lens 
only partially overlaps the field, then only the portion of the field accessible to the lens is 
printed with the photo. When the user moves the lens across the field the lens prints addi- 

» i*~no nicijr pivjiupi me usti iu move me iciib iu tuvci uu-pnuieu 

portions of the field. Any un-printed portions of the image are recorded as pending output 
against the page instance on the Netpage Network. If a lens comes into contact with the 
image field at a future time then any un-printed portions can be printed. 

If the user pastes an image into a lens-displayed image field when the lens is not in contact 
with the page, then the entire image is recorded as pending output. 

I he printing magic lens may incorporate a digital camera. Once the user takes a photo, it 
may float on the display as an icon until the user explicitly pastes it in an image field. The 
magic lens may provide dedicated buttons for browsing, viewing, editing and deleting a 
collection of photos stored in the lens. The tens may also support removable storage, e.g. 
via a memory card socket, for storage of photos. 

The user may paste personal information, such as contact details, into a Netpage form 
which is not otherwise linked to the user. 

5.6 Braille Magic Lens 

The Braille magic lens is analogous to the Netpage magic lens. It displays in registered 
fashion and via a tactile Braille display the text of a Netpage. ft also displays text label , 
corresponding to graphics and images, where available in the page description. Because 
Braille characters are rendered at larger sizes than \ isual \\ -recogn i/ed characters, the 
Braille magic lens also presents a zoomed view of the underK ing page. Ill is means that a< 
the magic lens is moved over the page, the display moves correspondingK faster 
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1 Self-Mapping Surfaces 



A self mapping surjace is a physical surface in the real world which contains a perfectly 
registered map of itself which can be queried by an appropriate sensing device A surface 
may be mapped sparsely or ubiquitously. The map may be encoded visibly or invisibly 
I he map may be defined at any required resolution. The map is defined in such a way that 
a local query on the mapped surface yields an unambigous map reference both within the 
map and among different maps. 

If a computer system contains information about features on a self-mapping surface then 
such information can be efficiently retrieved based on map references supplied by the 
sensing device. The information thus retrieved can be used in many ways. It can be sup- 
plied directly to the operator of the sensing device in the form of audio video text or 
graphics, e.ther projected onto the surface, virtually projected onto the surface' by sur- 
Iace-reg,stered display through the operator's head-mounted display, or displayed on a 
normal computer screen. It can, more generally, take the form of actions which are initi- 
ated by the computer system on behalf of the operator in response to .he nrvr^rv in- 
action with the surface feature. ' ' 

Self-mapping surfaces can be categorised according to whether the map is active or pas- 
sive ,.e. whether the map actively participates when it is queried or not. Digitizing tablets 
touch-sensmve screens and CRTs sensed by light pens are all examples of actively 
self.mapp.ng_ surfaces. Active maps are typically only partially self-mapping because 
they do not yield unambiguous inter-map references per se. 

A passive map requires an active sensing device to perform the map query. 

A surface map is encoded according to the following general principles: 

• the encoding consists of a tiling of tugs each of which encodes surface id and position 

• tags are sufficiently small so that a single button click is guaranteed to sense at least 

one full tag 

• tags arc sufficiently large to be able to be applied reliably to the surface usin<. available 
technology (e.g. printing), and to be able to sensed reliably (e.g. optically) " 

• tags encode their position on the page to a higher precision than the tag density 

• tags need not tile a region with uniform density 

• higher-precision position information is deduced from the position of the tasi in the 

sensed area 

• each tag contains an easily-recognized invariant structure „aruel) which „/.A /,„„.,/ 
detect,.,,,, and which aids in undoing the effect of any u„//> induced hy the surface or 

hy the sensing process 

• the tag target is rotaiionally asymmetric so that the page can be imaeed in any orienta- 
tion and the orientation can be determined 

• the tag target is fault-tolerant for immunity to surface damage and sensing errors 

• the tag data is fault-tolerant for immunity to surface damage and sensing errors 
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2 Computer System User 
Interface 

A passive self-mapping surface can support a graphical user interface to a computer sys- 
tem, mediated by a sensing pen: 

• encoding of position to support context-dependent button clicking 
drawing, writing and gesturing, or 

• encoding of object id to support context-free button clicking, or 

• encoding of page id and position to support context-free button clicking, 
drawing, writing and gesturing 

• invisible encoding to support clicking, drawing, writing and gesturing 
anywhere, without visible interference with surface features 

• fault-tolerant encoding/or immunity to surface damage and sensing errors 

Minimal interface: 

• consists of a physical page with a virtual page structure 

• page surface contains features, e.g. graphical user interface elements 

• features are typically printed onto the page surface 

• physical page surface can be any surface, including the surface of a sheet of paper 

• surface features are represented in corresponding page structure 
(there is a one-to-one mapping between the two) 

• physical page is a (passive) self-mapping surface 

• map uses fan It- tolerant optical encoding 

• encoding is typically printed onto the surface using invisible (c.n. IR) ink 

• map is sensed by wireless optically sensing pen or stylus 

• pen senses map when nib is close to or in contact with surface 

• pen senses map reference at sufficient rate to capture human handwrrtinu. 
i.e. at 200 dpi or greater and 100 11/ or faster [10] 

• pen allows user to indicate "pen down'" and "pen up" action. 

e.g. via button on shaft, or sprung button or pressure sensor coupled to nib 

• pen generates digital ink with embedded map references 

• digital ink consists of strokes 

• stroke consists of timestamped positions bracketed by pen down and pen up 

• pen transmits digital ink to base station 

• page structure is known to base station 

• base station interprets digital ink with respect to page structure, 
e.g. as operations on a graphical user interface 

Page structure: 

• page structure defines graphical elements and input elements 

• graphical elements include text, graphics and images 

• input elements include hyperlinks and form fields 

• base station interprets digital ink as element selections, 
form field fill-ins, and hyperlink activations 



25 November 1999 



Silverbrook Research 



Self-Mapping Surfaces Design Description 



draft vO 3 



Marking versus non-marking pen input: 

• user gets visible feedback on input because pen has a marking nib 

• pen also has a non-marking nib for command input (e.g. button presses) 

Network; 

• page structures are available on network 

• pages are uniquely identified within particular domain 

• base station retrieves page structures on demand by page id (from map reference) 

• hy perl inked objects are available on network 

• base station communicates with linked objects to handle hyperlink activations 
Base station is printer: 

• base station is a printer 

• prints hyperlinked pages on demand 

• prints subscribed pages autonomously 

Persistence: 

• page structures are automatically archived on network, 
making all published content persistent 

Handwriting recognition: 

• handwriting is automatically converted to text in text input Held 

• handwriting recognition uses dynamics in digital ink 

• unrecognized handwriting can be automatically routed to human operator 

Verified identity: 

• pen acts as identity token 

• signature is automatically verified in signature input field with respect to identity 

• signature verification uses dynamics in digital ink, including pressure 

• verified identity allows secure transaction authorization 
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3 Tag Tiling 

Paeh tag is entirely self-suf ficient. A tag can be located and decoded independently of sur- 
rounding information (e.g. other tags). Hach tag identifies the surface on which it lies, and 
each tag specifies its own position on the surface. How a set of tags are arranged on a sur- 
face is somewhat immaterial, so long as the surface is tiled sufficiently densely in areas 
where tags are required, e.g. in the active areas of buttons or form fill-in fields. A tag tiling 
is sufficiently dense if the tag sensor can image at least one tag on a single "click" within 
any active area on the surface. 

[here are two reasons for using an irregular tiling: 

• an irregular tiling is more easily mapped to a surface of arbitrary topography and 
topology 

• an irregular tiling is less susceptible to damage from systematic artifacts; for example, 
if a tiling is printed onto a surface, a stuck nozzle can produce a column of dots which 
are always on or always off 

An irregular tag tiling can be produced for a surface of arbitrary topography and topology 
as follows: 

(a) generate a Poisson-disk distribution of tag positions within an equilateral triangle of 
a certain size; ensure tag separation not just within the triangle, but also across the 
boundary between any two arbitrarily rotated adjacent copies of the trian izle in anv 
allowed triangle mesh 

(b) compute a regular (quasi-equilateral) triangle mesh which approximates the surface; 
a triangle mesh can approximate a surface of arbitrary topography and topoloizy 
without introducing discontinuities or singularities, although surface detail which is 
finer than the size of the triangle may have to be ignored 

(c) map the tag triangle to each triangle of the mesh 

(d) project each mapped tag triangle onto the surface 

(e) render each tag in each projected triangle on the surface 

This approach is based a pattern-based texturing approach described in [8]. That approach 
in turn relies on a surface re-tiling approach described in [11]. 

f igure I shows a regular tag triangle mesh which might be used to tan a planar surface 
such as a piece of paper. It also shows a distribution of eight tags within the tan triangle. 

I he process presumes that a polygonal model of the object exists and that rendering regis- 
tered tags onto its surface is possible. A polygonal model of an actual object can be cre- 
ated from range-scanning data. A polygonal model may also exist because the object was 
manufactured from a polygonal model. 1 he issue of rendering registered tans onto an arbi- 
trary surface is not considered any further here. 

A Poisson-disk distribution is a Poisson distribution which satisfies the Poisson-disk crite- 
rion f" 7 ! where the ^ e n; i r; in on of ;m\ tuo • un i - 1. • \- ^^\^r- |,> rh . rv. • . . j . ; 
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Figure 1. Tag triangle mesh and tag triangle 

Hach tag must now encode the following: 

• the id of the tag position within the tag triangle 

• the id of the tag triangle within the surface 

• the id of the surface 

A surface-specific description of the tag tiling can be retrieved based on the surface id. 
possible from a network. The description includes the following: 

• the shape and size of the tag triangle 

• the distribution of tag positions w ithin the tag triangle 

• the arrangement of tag triangles on the surface 

A particular tag tiling may be used for a class of surfaces, e g. all sheen of I .etter-si/ed 
paper, all double-donut Zap widgets, etc. If the tiling is regular in nature, e.g. if the surface 
is planar, then the triangle id can typically be translated into a surface position algorithm i- 
cally rather than by table lookup. 

Sensed tag information can be translated into a high-precision surface position as follows: 

• translate tag triangle id into surface position 

• translate tag position id into offset and add to surface position 

• add offset of tag in imaged area to surface position 

lag tilings using non-triangular polygonal mesh units are also possible. Rectangular and 
hexagonal tilings of a plane are obviously possible, for example. Multiple different polyg- 
onal mesh units can also be used in a tiling, so long as they are identified in the tiling 
description. Multiple tag position distributions can also be used in a tiling, so long they are 
identified in the tiling description. 

When a tag triangle is mapped onto a mesh triangle, distortion may occur because the 
mesh triangle is only <///c/.w-equi lateral and may vary slightly in size from the tag triangle. 
[ urthennore, when the mesh triangle is projected onto the underlying surface, distortion 
ma> occur because of the greater detail of the surface. The first kind of distortion can be 
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reduced by computing tag distributions for a set of triangles of different shapes, and using 
the distribution for the shape which best fits the mesh triangle. The second kind of distor- 
tion can be reduced by using smaller tag triangles, although a tag triangle must always be 
large enough to support a proper Poisson-disk distribution. Alternatively, each triangle in 
the tag tiling description can be tagged with its distortion, so that the effects of the distor- 
tion can be included when a tag is interpreted. 

However, if the surface features to which the tags are intended to relate are projected and 
rendered onto the surface using the same mesh, then the features will be distorted in the 
same way as the tags, and the features and tags will remain registered to each other. 

No matter what approach is taken to handling distortion, it is important that the distortion 
does not cause the tag density to drop below the required threshold in any area, and con- 
versely, that it does not cause tag overlap. If tags are found to overlap, then one of the 
offending tags can be removed, so long as this does not adversely affect tag density. 

As stated above, when a Poisson-disk distribution of tag positions is generated for the tag 
triangle, tag separation must be enforced across the boundary between any two arbitrarily 
rotated adjacent copies of the triangle in any allowed triangle mesh. Figure 2 illustrates the 
number of unique triangle pair arrangements in an equilateral triangle mesh. For the pur- 
poses of ensuring tag separation there are six unique edge-to-edge arrangements. If the 
mirror image of the tag triangle is also used, then there are twenty-four unique arrange- 
ments. Vertex-to-vertex arrangements don't need to be considered because a tag will repel 
itself across an edge before it can repel itself across a vertex. 




Figure 2. Number of unique triangle pair arrangements, 
without and with mirroring (figures in brackets) 

During tag capture, the tag sensor must guarantee to capture at least one full tag when 
imaging an\ point on the surface. If tags are tiled on a regular grid then the tag sensor must 
image a circular area large enough to encompass the unit cell of the grid The radius of the 
tag must also be added to the radius of the imaged area. If the tags are jittered on the grid, 
then the maximum jitter radius must also be added to the radius of the imaged area. I 'll is in 
illustrated for a triangular grid in Figure 3. A triangular grid provides the best tag packing 
of all regular grids. 

It the center-to-center tag distance on a uniform triangular grid is A, the tag radius is /\ and 
the jitter radius is /. then the radius m of the circular area which must be imaged b\ the 
sensor is given bv: 
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4 Tag Capture 

The sensor used to image and decode position tags on a self-mapping surface is typically 
in the shape of a pen or st \ I us 

There is wide variation m the tilt at which the pen is operated by different users and even 
by the same user at different times. This complicates the optics in the pen and the subse- 
quent position tag decoding 

A reasonable tilt range needs to be supported, e g longitudinal tilt in the range 0° to 60°, 
and lateral tilt in the range -45° to +4Y When the pen is vertical, i.e. normal to the paper 
surface, it has zero tilt f igure 4 illustrates the lateral and longitudinal tilt planes. 



▲ 




Figure 4. Lateral and longitudinal pen tilt 

Assuming simple fixed optics, tilt introduces two problems: 

• out-of- focus blurnnu 

• perspective distortion 

The degree of bin mm- depend-, on the depth ot- field of the optics. 

Perspective distortion makes position tag delect ion more difficult. Conceptually, the tiit 
must somehow be interred to allow the peispcui\e distortion to be removed before fun her 
image processing takes place Hgure 5 illustrates tag perspective distortion. 



Figure 5. Tag perspective distortion 
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Figure 6. Tilt-induced effects 



Figure 6 illustrates the effect of tilt on depth, perspective distortion and displacement in 
one dimension. D is the nominal distance from the lens plane to the paper surface when 
the pen is vertical; d is the actual tilt-induced distance to the center of the imaged area (B); 
a is the width of the imaged area; .v is the distance from the pen axis to the imaging axis; 0 
is the pen tilt. 

Ihree tilt-induced effects are of interest: the change in distance to the paper surface, the 
perspective distortion of the imaged area, and the change in the position of the center of 
the imaged area, relative to the origin (i.e. the pen nib position). 

1 lie ti It-induced distance to the paper is given bv: 

d = D - s( cotO) (EQ 2) 

it /> is large compared with .v, this effect is less significant. It needs to be quantified rela- 
tive to the depth-of-field of the optics 

I he perspective distortion of the imaged area, given by the ratio of imaged feature si/es at 
points A and C on the paper surface, is given by: 



d - a 



(EQ 3) 



Where: 



-v = |(cot()) (EQ 4) 

If D is large compared with a, this effect is less significant. It needs to be quantified rela- 
tive both to the depth-of-field of the optics, and to the size of the position tag and the 
required image sensor resolution 
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The change in position of the center of the imaged area (B) relative to the origin {()) is 
given by: 



This effect is always significant for non-trivial tilt. 

Naive tilt sensing based on gravity, e.g using passive accelerometers, is inadequate 
because the page itself may be significantly tilted when the pen is being operated. Remem- 
ber that the page is simply a piece of paper, not a tabletop electronic tablet. 

Out-of-focus blurring can be eliminated by having variable-focus optics and an 
auto- focusing mechanism, e.g. a lens ser\u driven by image sharpness. 

Tilt can be inferred in several ways. It can be computed from the perspective distortion 
observed of the otherwise rectangular position tag target areas. This is non-trivial because 
the targets have a complex structure which is potentially difficult to detect when they are 
distorted This can be ameliorated bv intp.*ducin a additional structure into the overall ta^ 
tiling which is easier to detect than the tag targets themselves, although this further 
reduces the tag resolution. Tilt can also he computed from the distortion observed of a 
simpler pattern of light projected onto the page area being imaged. 

The projected pattern must contain both longitudinal and lateral structure to allow both 
longitudinal and lateral tilt to be determined. The simplest pattern consist of four separated 
spots, two on the longitudinal axis and tun on the lateral axis. Tilt along each axis is com- 
puted separately from the observed change in the c» >rres ponding inter- spot distance. This 
is illustrated in Figure 7. 



.v(secO- I) 



(EQ 5) 



f , r 



I, >n H it u. \\ r .'I: 



It^itlJ.JlMjl M 



Figure 7. Tilt-induced spot pattern perspective distortion 
(arrows show direction of tilt, 
dashed lines indicate changed inter-spot distance) 
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The position tag macrodot pattern contains mostly high frequencies, and so reduces to a 
uniform gray if sufficiently low-pass filtered. If a tilt pattern spot is sufficiently large com- 
pared with the size of a macrodot, then it is easy to separate from the background of the 
position tag data. Once a spot is located, its center can be computed to greater accuracy. 

As an alternative to projecting a spot pattern, each tag can include a structure which serves 
the same purpose. This structure should ideally project outside the tag body, to maximise 
its size and therefore maximise the accuracy with which perspective distortion is mea- 
sured. Adjacent tags can have interleaved projections so long as they are non-overlapping, 
as illustrated in Figure 8 



Figure 8. Interleaved tags with external perspective targets 

Once one spot is found, its partner is found along the same longitudinal or lateral axis. 
I his makes detection mure efficient. Similarly, since tilt typically changes slow ly over 
time, the last known spot positions provide a good starting point for detection. At a more 
sophisticated level, longitudinal tilt and lateral tilt angular velocities can be maintained 
over time, allow ing more accurate prediction of spot positions. 

Perspective distortion can be eliminated entirely if the capture lens is always kept parallel 
to the page surface. Rather than achieving this using a moving lens, an alternative 
approach uses a w ide-angle lens which always captures some part of the paper surface 
without distortion, as illustrated in Figure 9. 



•y 




paper at 45° pen U\t 
/ ^ paper at 30° pen r.ifr 
paper at V:'" pen tilt 
. paper at G° pen U\t 

Figure 9. Undistorted imaging through wide-angle lens 

I he lens system can be further refined to provide different focal lengths at different posi- 
tions, tuned to the expected distance to the page at the corresponding tilt. This may elimi- 
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natc the need for an auto-focus mechanism. However, since the same part of the lens will 
be imaging a s,gn,fleantly smaller depth when imaging the left side of the area of interest 
than when imaging the right side, the lens system may still provide insufficient 
dcpih-of-field to eliminate the need for variable focus. 

The cost of this approach is that a larger area must be imaged, requiring a larger image 
sensor. It does not, however, place an additional burden on tilt decoding, since the tilt 
spots are restricted to a sub-area of the image area. Similarly, it does not place an addi- 
tional burden on tag decoding, since once the tilt is known, the relevant tag sub-area is 

known. te 

The variable-focus wide-angle lens system can be designed using an automatic optimiza- 
tion technique such as a genetic algorithm. 
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Applications 
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5 Digitizer 



5.1 Large-Format Digitizer 



A passive self-mapping surface provides the basis for a large- format high-resolution digi- 
tizing tablet for a screen-based graphical user interface The "digitizer" surface can be 
printed onto contact plastic which can then be glued to any surface, including the entire 
surface of a desk. The digitizer can be sold in sheets which can be used to tile a surface. 
The desk surface can itself be printed as a digitizer. The tags can be protected from wear 
by a tough overcoat or transparent laminate. 

An imaging pen produces a stream of digital ink when interacting with the surface. The 
pen can be wired or wireless. A wireless pen may be able to communicate with a standard 
Bluetooth transceiver in the workstation in PC. 

The user can man the larpe-format H 

, ^ . ^ . „ ~ . fc , . , , , t ^ i ^.Aitiiutu viuudi wui Nbpace Wll icn 

exceeds the size of the workstation or PC screen. The system can be configured to ensure 
that the point of focus on the digitizer is always displayed on the screen, i.eTthe screen can 
follow the pen. 



5.2 Dynamic Handwriting Capture 

A passive self-mapping surface provides the basis for a low-cost high-resolution di^itizin- 
tablet for dynamic handwriting capture in conjunction with an imaging pen. 

5.3 Dynamic Signature Capture 

A passive self-mapping surface provides the basis for a low-cost high-resolution di-itizin- 
tablet for dynamic handwriting capture in conjunction with an imaging and pressure-sens^ 
ing pen. 

5.4 Registered Projection 

A self-mapping surface can be used as a large low-cost digitizing tablet for a screen-based 
graphical user interface system to a personal computer or workstation. If the screen output 
of the computer system is projected onto the self-mapping surface, in perfect registration 
with the surface map, then the surface becomes an interactive drawing surface, where 
drawing mediated In computer, it can thus be used for all kinds of interactive modelling 
design, draw ing. painting, etc. 

I he projector can include an imaging unit which senses the large-scale map of the surface 
(if it is encoded at multiple resolutions) and uses this information to control a servo which 
registers the projected image with the surafce map. 

Such a surface supports easy multi-user participation and can support the ultimate active 
desktop. 



Stlverbrook Research 



Self-Mapping Surfaces Design Description 



draft vO 3 



6 Situated User Interfaces 

6.1 Device Control User Interface 

There are many electrical and electronic devices in the home and office which have some 
kind of user interface for controlling their operation. These include office equipment, 
audio and video equipment, kitchen appliances, heating and cooling systems, etc. Even 
light sockets have on/off switches and sometimes dimmers, and in many countries power 
outlets have on/off switches. 

Office equipment and entertainment equipment is increasingly networkable - i.e. devices 
come with interfaces to standard local-area networks, allowing their operation to be con- 
trolled over a network. The advent of standard wireless networking technologies such as 
Bluetooth may see an increase in networking support in such devices and appliances. 

Some networking futurists predict that even light sockets will one day be networked, 
allowing their status to be queried and their function to be controlled remotely. 

If a particular device is networked, and the network interface of the device allows the 
device to be fully controlled, then it is possible to completely separate the user interface to 
the device from the device itself. Furthermore, the user interface can then be implemented 
— ^ p«^> ^i.-M.appii^ oui iav.L, Jtiutu uy a wirciess sensing device carried Dy tne user. 
The sensing device must be small, unobtrusive and portable. It might be embedded in a 
ring, glove or finger glove worn by the user. It could also be embedded in custom eyewear 
with interaction controlled by gaze. 

The self-mapping surface is coded with the unique id of the device, and is printed with the 
user interface of the device in the form of buttons etc. Being passive, the surface gives no 
feedback to the user as to the status of the device or its user interface. The base station for 
the sensing device carried by the user resolves operations on the self-mapping surface 

with reference to a user interface structure previously selected by the user from the device 
itself, or designed by the user using elements provided bv the device. 

User interfaces implemented in this way have several advantages: 

• once the infrastructure is in place, the cost of manufacturing the user interface is 
removed from the cost of manufacturing each particular device; the cost of the user 
interface to a light socket, for example, is non-trivial 

• a user interface can be replicated by simply printing another copy 

• when a user interface becomes damaged it can be easily replaced 

• when the firmware in a device is updated, a new user interface can be easily provided 

• a user interface can be customized by arranging elements and printing the result 

• a user interface with a particular style can be chosen from an available set 

• multiple user interfaces can be grouped together 

• a user interface can be provided on the device itself by the same means 

• access to individual user interfaces and functions can be controlled, 
since each sensing device is tied to a particular user's identity 



25 November 1999 



Silverbrook Research 



Self-Mapping Surfaces Design Description 



draft vO 3 



6.2 User Interface on Toys 

The surface of a toy can be printed inexpensively with the user interface to the toy as a 
self-mapping surface. An appropriate sensing device can the be used to interact with the 
user interface, communicating by wire or wirelessly to either a personal computer, game 
console, or a processor embedded in the toy itself, to produce graphical output, sound 
effects and other behavior. 

A toy sit-in submarine may have a large user interface containing dozens of submarine 
controls. The toy comes with a CD-ROM containing an associated interactive game which 
runs on a PC or game console. The game is driven by the self-mapping surface user inter- 
face contained in the submarine, perhaps mediated by a sensing glove which communi- 
cates wirelessly by Bluetooth with standard Bluetooth hardware in the PC or game 
console. The PC screen or console-connected television monitor provides the animated 
part of the user interface. The self-mapping surface provides the extended user interface. 

Toys can be customized by selecting different user interfaces and printing them via a 
Netpage system [2]. Custom user interfaces can be created by assembling pre-defmed user 
interface components. 

6.3 Defect Logging 

The entire surface of an object, e.g. a Boeing 747 aircraft, can be printed as a self-mapping 
surface. During maintenance of the object, an engineer can register the precisely location 
of detected defects or parts requiring maintenance, simply by touching the surface with an 
appropriate sensing device. 
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7 Hybrid User Interfaces 

7.1 Hybrid User Interface to Web Browser 

Self-mapping surfaces containing printed Web pages can be used to request linked Web 
pages for display in a screen-based Web browser The printed Web pages provide an 
extended user interface to the Web. Printed pages in magazines can also be encoded as 
self-mapping surfaces so that their printed Web hyperlinks are also active. 

A single page id can be used per page, or individual object ids can be used for each 
embedded hyperlink. The latter is more practical, since object ids just have to be translated 
to Web URLs, probably by some network server. In the former case, someone needs to 
keep track of the printed layout and logical content of each page. If the Netpage Web 
printing service is used [2], then this comes for free of course 

All that is needed is a special pen which talks to a Web browser rather than the printer, or 
a normal pen which can be placed in this mode. 

7.2 Hybrid User Interface to Computer Application 

The user interface to a GUI-based computer application running on a multi-tasking com- 
puter can be printed as a self-mapping surface on a user interface or command "card 11 . The 
printed user interface can include a "digitizer pad" area for moving the GUI pointer rela- 
tive to the application. Invoking any function of the application^ user interface or moving 
the GUI pointer, automatically makes the application current - i.e. if the application is run- 
ning in a Windowed GUI system then its window is brought to the front and made current. 
If the application is not currently running, then it is automatically launched. 

The printed user interface for a text-oriented application can contain a printed keyboard, a 
general-purpose handwriting input text field, or both. 

A personal computer system or workstation can thus potentially consist of a screen for dis- 
playing GUI output, a number of application-specific printed user interfaces, a sensing 
device (typically a stylus) for sensing user operations relative to the printed user inter- 
faces, and a computer which receives wired or wireless transmissions from the sensing 
device, runs applications, and interprets sensed inputs relative to each application. 

Each printed user interface "card" can be encoded with a unique page id specific to the 
application, and tagged with an attribute which instructs the personal computer or work- 
station to interpret operations on the page relative to a local instance of the application, 
even in a global networked Netpage environment. 

If the computer is a network terminal connected to a LAN, an intranet, or the Internet, any 
interaction with the printed user interface launches or interacts with a networked instance 
of the application. 
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Hardware 
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8 Tag Sensor 



8.1 



Wearable Sensor 



A sensing device for a self-mapping surface worn on the body, e.g. in the shape of a ring, 
armband, bracelet, thimble, or mounted at the end of one of the fingers of a glove, or 
attached to a keyring, etc., can be used to conveniently interact with a self-mapping sur- 
face. It can contain an active sensor for sensing the area of the self-mapping surface with 
which it is brought into contact or proximity, and transmits digital ink by wire or wire- 
lessly to an associated base station, e.g. via Bluetooth. 
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